[jira] [Created] (MAHOUT-1921) DSSVD Propagates cache hint

[Trevor Grant (JIRA)]

Trevor Grant created MAHOUT-1921:
------------------------------------

Summary: DSSVD Propagates cache hint
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Priority: Minor
Fix For: 0.13.0


The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.

The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.

https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1921:
-----------------------------------
Sprint: Jan/Feb-2017

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Musselman (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Musselman reassigned MAHOUT-1921:
----------------------------------------

Assignee: Trevor Grant

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Assignee: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852594#comment-15852594 ]

ASF GitHub Bot commented on MAHOUT-1921:
----------------------------------------

GitHub user rawkintrevo opened a pull request:

https://github.com/apache/mahout/pull/274

MAHOUT-1921 Propagate cacheHint in dssvd

Propogate cache hint in dssvd

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rawkintrevo/mahout mahout-1921

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mahout/pull/274.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #274

----
commit 956f12d5b730a9ddb7bfd9ac857f3c9799b3cacc
Author: rawkintrevo <***@gmail.com>
Date: 2017-02-04T04:55:52Z

MAHOUT-1921 Propagate cacheHint in dssvd

----

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Assignee: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Trevor Grant (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on MAHOUT-1921 started by Trevor Grant.
--------------------------------------------

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Assignee: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852972#comment-15852972 ]

ASF GitHub Bot commented on MAHOUT-1921:
----------------------------------------

Github user andrewpalumbo commented on a diff in the pull request:

https://github.com/apache/mahout/pull/274#discussion_r99477184

--- Diff: math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala ---
@@ -22,13 +22,18 @@ object DSSVD {
* @return (U,V,s). Note that U, V are non-checkpointed matrices (i.e. one needs to actually use them
* e.g. save them to hdfs in order to trigger their computation.
*/
- def dssvd[K](drmA: DrmLike[K], k: Int, p: Int = 15, q: Int = 0):
+ def dssvd[K](drmA: DrmLike[K],
+ k: Int,
+ p: Int = 15,
+ q: Int = 0,
+ cacheHint: CacheHint.CacheHint = CacheHint.MEMORY_ONLY):
--- End diff --

Not sure what we should make the default `CacheHint` here and other places, e.g. #276.

for `drmDfsRead(...)` we use `CacheHint.NONE` but that may be something that we should let the user supply as well. Most parallaization methods use `CacheHint.NONE`. I would think that if we have anything hardcoded or as a default, we should use `CacheHint.MEMORY_AND_DISK` since we have no idea of the size of the user's data.. but I suppose that is making Assumptions as well.

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Assignee: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ASF GitHub Bot (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855266#comment-15855266 ]

ASF GitHub Bot commented on MAHOUT-1921:
----------------------------------------

Github user asfgit closed the pull request at:

https://github.com/apache/mahout/pull/274

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Assignee: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Trevor Grant (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Trevor Grant resolved MAHOUT-1921.
----------------------------------
Resolution: Fixed

Post by Trevor Grant (JIRA)
DSSVD Propagates cache hint
---------------------------
Key: MAHOUT-1921
URL: https://issues.apache.org/jira/browse/MAHOUT-1921
Project: Mahout
Issue Type: Improvement
Components: Math
Affects Versions: 0.13.0
Reporter: Trevor Grant
Assignee: Trevor Grant
Priority: Minor
Labels: beginner
Fix For: 0.13.0
The DSSVD does lots of check pointing, but currently only the default checkpoint cacheHint is given.
The user should be able to pass a checkpointing hint as this can lead to dramatic performance issues in some cases.
https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)