[jira] [Created] (MAHOUT-1939) fastutil version clash with spark distributions

[Dmitriy Lyubimov (JIRA)]

Dmitriy Lyubimov created MAHOUT-1939:
----------------------------------------

Summary: fastutil version clash with spark distributions
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical


Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).

observed version in CDH:

file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar

mahout uses 7.0.12

java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Dmitriy Lyubimov (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862045#comment-15862045 ]

Dmitriy Lyubimov commented on MAHOUT-1939:
------------------------------------------

perhaps mahout should include fast-util in a shaded form in mahout-math or mahout-math-scala.

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Dmitriy Lyubimov (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862045#comment-15862045 ]

Dmitriy Lyubimov edited comment on MAHOUT-1939 at 2/11/17 12:16 AM:
--------------------------------------------------------------------

perhaps mahout should include fast-util in a shaded form in mahout-math or mahout-math-scala.
like this (in mahout-math):

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>it.unimi.dsi.fastutil</pattern>
<shadedPattern>shaded.it.unimi.dsi.fastutil</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>



was (Author: dlyubimov):
perhaps mahout should include fast-util in a shaded form in mahout-math or mahout-math-scala.

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Dmitriy Lyubimov (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862045#comment-15862045 ]

Dmitriy Lyubimov edited comment on MAHOUT-1939 at 2/11/17 12:33 AM:
--------------------------------------------------------------------

perhaps mahout should include fast-util in a shaded form in mahout-math or mahout-math-scala.
like this (in mahout-math):

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<includes>
<include>it.unimi.dsi:fastutil</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>it.unimi.dsi.fastutil</pattern>
<shadedPattern>shaded.it.unimi.dsi.fastutil</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>



was (Author: dlyubimov):
perhaps mahout should include fast-util in a shaded form in mahout-math or mahout-math-scala.
like this (in mahout-math):

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>it.unimi.dsi.fastutil</pattern>
<shadedPattern>shaded.it.unimi.dsi.fastutil</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863101#comment-15863101 ]

Andrew Palumbo commented on MAHOUT-1939:
----------------------------------------

[~dlyubimov] How many CDH Spark versions does this break? Is this a blocker for the upcoming release? Seems so.

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Dmitriy Lyubimov]

I don't know exactly. That current more or less recent version is broken
(with spark 1.6.0) it seems.

Post by Dmitriy Lyubimov (JIRA)
[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=
com.atlassian.jira.plugin.system.issuetabpanels:comment-
tabpanel&focusedCommentId=15863101#comment-15863101 ]
----------------------------------------
[~dlyubimov] How many CDH Spark versions does this break? Is this a
blocker for the upcoming release? Seems so.

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical
Version difference in fast util breaks sparse algebra (specifically,

RandomAccessSparseVector in assign, e.g., vec *= 5).

Post by Dmitriy Lyubimov (JIRA)
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/

jars/fastutil-6.3.jar

Post by Dmitriy Lyubimov (JIRA)
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$

BasicEntry.setValue(AbstractInt2DoubleMap.java:146)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.RandomAccessSparseVector$

RandomAccessElement.set(RandomAccessSparseVector.java:235)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.VectorView$DecoratorElement.

set(VectorView.java:181)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.AbstractVector.assign(

AbstractVector.java:536)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$

div$eq(RLikeVectorOps.scala:45)

Post by Dmitriy Lyubimov (JIRA)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1939:
-----------------------------------
Fix Version/s: 0.13.0

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Blocker
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1939:
-----------------------------------
Priority: Blocker (was: Critical)

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Blocker
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1939:
-----------------------------------
Sprint: Jan/Feb-2017

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Blocker
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Dmitriy Lyubimov]

fwiw i am not sure if shading fastutils is generally the best approach, as
it blows the size of the math-scala 10x, but i did verify it worked with
cdh.

Post by Dmitriy Lyubimov (JIRA)
[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=
com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
-----------------------------------
Sprint: Jan/Feb-2017

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Blocker
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically,

RandomAccessSparseVector in assign, e.g., vec *= 5).

Post by Dmitriy Lyubimov (JIRA)
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/

jars/fastutil-6.3.jar

Post by Dmitriy Lyubimov (JIRA)
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$

BasicEntry.setValue(AbstractInt2DoubleMap.java:146)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.RandomAccessSparseVector$

RandomAccessElement.set(RandomAccessSparseVector.java:235)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.VectorView$DecoratorElement.

set(VectorView.java:181)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.AbstractVector.assign(

AbstractVector.java:536)

Post by Dmitriy Lyubimov (JIRA)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$

div$eq(RLikeVectorOps.scala:45)

Post by Dmitriy Lyubimov (JIRA)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1939:
-----------------------------------
Priority: Critical (was: Blocker)

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo reassigned MAHOUT-1939:
--------------------------------------

Assignee: Andrew Palumbo

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on MAHOUT-1939 started by Andrew Palumbo.
----------------------------------------------

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884592#comment-15884592 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

GitHub user andrewpalumbo opened a pull request:

https://github.com/apache/mahout/pull/285

MAHOUT-1939: Shade fastutil jar conflicictng with CDH Spark

Reports of fastutils versions conflicting on CDH. Shade our fastutils in `mahout-math`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewpalumbo/mahout MAHOUT-1939

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mahout/pull/285.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #285

----
commit 4095c32e99b69f48ff5e091d612eb65569c25f73
Author: Andrew Palumbo <***@apache.org>
Date: 2017-02-26T07:25:35Z

MAHOUT-1939: Shade fastutil jar conflicictng with CDH

----

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884597#comment-15884597 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user andrewpalumbo commented on the issue:

https://github.com/apache/mahout/pull/285

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo resolved MAHOUT-1939.
------------------------------------
Resolution: Fixed

committed in 4d1464ef4d7a1d278c828d29a906e8f431a2fcb5

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885066#comment-15885066 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user andrewpalumbo closed the pull request at:

https://github.com/apache/mahout/pull/285

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885065#comment-15885065 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user andrewpalumbo commented on the issue:

https://github.com/apache/mahout/pull/285

committed in 4d1464ef4d7a1d278c828d29a906e8f431a2fcb5

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Hudson (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885091#comment-15885091 ]

Hudson commented on MAHOUT-1939:
--------------------------------

FAILURE: Integrated in Jenkins build Mahout-Quality #3428 (See [https://builds.apache.org/job/Mahout-Quality/3428/])
MAHOUT-1939: Shade fastutil jar conflicictng with CDH Spark #285 (apalumbo: rev 4d1464ef4d7a1d278c828d29a906e8f431a2fcb5)
* (edit) math/pom.xml

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886709#comment-15886709 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user andrewpalumbo commented on the issue:

https://github.com/apache/mahout/pull/285

+1 shipit ! thx!

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903237#comment-15903237 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

GitHub user andrewpalumbo opened a pull request:

https://github.com/apache/mahout/pull/293

[MAHOUT-1939] Revert for CLI Driver Fix

fastutils shaded jar was creating an unwanted dependency-reduced-jar in mahout math. and not being passed to CLI or shell.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewpalumbo/mahout MAHOUT-1939a

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mahout/pull/293.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #293

----
commit 96c3ad4df614ad31004e5c3cd6e115245c88fc44
Author: Andrew Palumbo <***@apache.org>
Date: 2017-03-09T15:38:34Z

[MAHOUT-1939] Revert for CLI Driver Fix

----

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903292#comment-15903292 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user andrewpalumbo commented on the issue:

https://github.com/apache/mahout/pull/293

All tests pass

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903338#comment-15903338 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user asfgit closed the pull request at:

https://github.com/apache/mahout/pull/293

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Hudson (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903444#comment-15903444 ]

Hudson commented on MAHOUT-1939:
--------------------------------

SUCCESS: Integrated in Jenkins build Mahout-Quality #3445 (See [https://builds.apache.org/job/Mahout-Quality/3445/])
[MAHOUT-1939] Remove fastutils shaded jar. Not being picked up by CLI (apalumbo: rev b32b6d9731481f00ea3b96e3129b9bc96e34eff5)
* (edit) math/pom.xml

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905383#comment-15905383 ]

ASF GitHub Bot commented on MAHOUT-1939:
----------------------------------------

Github user andrewpalumbo commented on the issue:

https://github.com/apache/mahout/pull/293

Yeah I should have deleted.. at the last moment I decided to leave them in in case we needed them somewhere else.



Sent from my Verizon Wireless 4G LTE smartphone


-------- Original message --------
From: Andrew Musselman <***@github.com>
Date: 03/09/2017 7:43 AM (GMT-08:00)
To: apache/mahout <***@noreply.github.com>
Cc: Andrew Palumbo <***@outlook.com>, Author <***@noreply.github.com>
Subject: Re: [apache/mahout] [MAHOUT-1939] Revert for CLI Driver Fix (#293)


@andrewmusselman commented on this pull request.

Cool, can we delete this instead of commenting?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://github.com/apache/mahout/pull/293#pullrequestreview-26061794>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AHU2HZH_3DYx8TsUqfCvY7GjW9HB1Ulqks5rkB4ogaJpZM4MYP3f>.

Post by Dmitriy Lyubimov (JIRA)
fastutil version clash with spark distributions
-----------------------------------------------
Key: MAHOUT-1939
URL: https://issues.apache.org/jira/browse/MAHOUT-1939
Project: Mahout
Issue Type: Bug
Reporter: Dmitriy Lyubimov
Assignee: Andrew Palumbo
Priority: Critical
Fix For: 0.13.0
Version difference in fast util breaks sparse algebra (specifically, RandomAccessSparseVector in assign, e.g., vec *= 5).
file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/jars/fastutil-6.3.jar
mahout uses 7.0.12
java.lang.UnsupportedOperationException
at it.unimi.dsi.fastutil.ints.AbstractInt2DoubleMap$BasicEntry.setValue(AbstractInt2DoubleMap.java:146)
at org.apache.mahout.math.RandomAccessSparseVector$RandomAccessElement.set(RandomAccessSparseVector.java:235)
at org.apache.mahout.math.VectorView$DecoratorElement.set(VectorView.java:181)
at org.apache.mahout.math.AbstractVector.assign(AbstractVector.java:536)
at org.apache.mahout.math.scalabindings.RLikeVectorOps.$div$eq(RLikeVectorOps.scala:45)
...

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)