[jira] [Created] (MAHOUT-1904) Create a test harness to test mahout across different hardware configurations

[Andrew Palumbo (JIRA)]

Andrew Palumbo created MAHOUT-1904:
--------------------------------------

Summary: Create a test harness to test mahout across different hardware configurations
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Fix For: 0.13.0


Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Sprint: Jan/Feb-2017

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Assignee: Pat Ferrel

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Pat Ferrel
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Assignee: (was: Pat Ferrel)
Labels: test (was: )

@pat- assigning this to you since you have the least issues assigned for this release, and I'm guessing that you have some code laying around for this- If you cant pick this Up, Maybe [~andrew.musselman] can pick it up?

We basically need to test Drm %*% Drm, Vec%*% Drm, transposes, etc, large and small sizes, and a timer. A simple output (eg. Test name , sizes if it is just simple MMul, etc and time to process) to stdout would be good also, which we could pipe to files to generate reports.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Pat Ferrel (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822907#comment-15822907 ]

Pat Ferrel commented on MAHOUT-1904:
------------------------------------

Did you have in mind a CLI tool or unit test? I assume the former since this should be runnable on various clusters and configs? Is this meant to be a benchmark?

Seems like an example maybe rather than CLI in mahout itself.

Not sure I have enough time for 0.13.0 and no don't have a good test harness. If we move this out to the next release I'd be interested in doing it since recently I've become more interested in performance.

[~Andrew_Palumbo] could you supply some examples? We have at least one medium sized dataset of 2 matrices (the dating site, can't recall the name) for the larger tests but still they are small compared to real-world data.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Pat Ferrel (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pat Ferrel updated MAHOUT-1904:
-------------------------------
Affects Version/s: (was: 0.12.2)
0.14.0

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822915#comment-15822915 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~pferrel] I had done some similar work to this JIRA in the past few months, namely created a very simple test harness that evaluated the operation of some ML algorithms using a scala timer and wrote out a report file, not sure if or why that work was not accepted at that point, regardless I'd like to help you with this effort and see if we can leverage any of the infrastructure that I had created, please let me know if thats ok.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822915#comment-15822915 ]

Saikat Kanjilal edited comment on MAHOUT-1904 at 1/14/17 7:50 PM:
------------------------------------------------------------------

[~pferrel] I had done some similar work to this JIRA in the past few months, namely created a very simple test harness that evaluated the operation of some ML algorithms using a scala timer and wrote out a report file, not sure if or why that work was not accepted at that point, regardless I'd like to help you with this effort and see if we can leverage any of the infrastructure that I had created, please let me know if thats ok.

Here's the github repo branch: https://github.com/skanjila/mahout/tree/mahout-1869


was (Author: kanjilal):
[~pferrel] I had done some similar work to this JIRA in the past few months, namely created a very simple test harness that evaluated the operation of some ML algorithms using a scala timer and wrote out a report file, not sure if or why that work was not accepted at that point, regardless I'd like to help you with this effort and see if we can leverage any of the infrastructure that I had created, please let me know if thats ok.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Affects Version/s: (was: 0.14.0)
0.13.0
0.12.2

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Assignee: Suneel Marthi

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Suneel Marthi
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo reassigned MAHOUT-1904:
--------------------------------------

Assignee: Andrew Palumbo (was: Suneel Marthi)

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Assignee: Suneel Marthi (was: Andrew Palumbo)

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Suneel Marthi
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826470#comment-15826470 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~Andrew_Palumbo] I didn't hear back from you, can I work with you to explore/hack at the solution I created earlier to drive this forward, if so I can working on these tests, I would love to eventually build out a performance dashboard within mahout to do this , I can discuss on dev list with a proposal if you're interested.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Suneel Marthi
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826470#comment-15826470 ]

Saikat Kanjilal edited comment on MAHOUT-1904 at 1/17/17 5:43 PM:
------------------------------------------------------------------

[~Andrew_Palumbo] Since this is now assigned to you, can I work with you to explore/hack at the solution I created earlier to drive this forward, if so I can working on these tests, I would love to eventually build out a performance dashboard within mahout to do this , I can discuss on dev list with a proposal if you're interested.


was (Author: kanjilal):
[~Andrew_Palumbo] I didn't hear back from you, can I work with you to explore/hack at the solution I created earlier to drive this forward, if so I can working on these tests, I would love to eventually build out a performance dashboard within mahout to do this , I can discuss on dev list with a proposal if you're interested.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Suneel Marthi
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826528#comment-15826528 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

ok guess we're going back and forth on this [~smarthi] can I help on this, see my comments above.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Suneel Marthi
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833977#comment-15833977 ]

Andrew Palumbo commented on MAHOUT-1904:
----------------------------------------

This is not intended as a benchmark, simply a set of basic applications that we can run to ensure that the release is configured correctly and runs for each package. Spark only applications would be great- even a set of .mscala scripts that we can load into the shell for each hardware build; simply to ensure that the build is working on that set of hardware.

The tests that we do with every release

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.13.0, 0.12.2
Reporter: Andrew Palumbo
Assignee: Suneel Marthi
Labels: test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848761#comment-15848761 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~Andrew_Palumbo] I see this script: https://github.com/apache/mahout/blob/master/examples/bin/SparseSparseDrmTimer.mscala, is this an example of the type of thing we need to emulate but add reporting capability to this? I was going to start with this script and add the reporting aspect to it if that makes sense.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849134#comment-15849134 ]

Andrew Palumbo commented on MAHOUT-1904:
----------------------------------------

Good call [~kanjilal] thank you for reminding me of that.

The only real reporting that we need is pass/fail so that script with more functions makes a good start. I'm assigning this to myself, but please feel free to do some work on it and submit a PR.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo reassigned MAHOUT-1904:
--------------------------------------

Assignee: Andrew Palumbo

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on MAHOUT-1904 started by Andrew Palumbo.
----------------------------------------------

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Priority: Critical (was: Major)

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850809#comment-15850809 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~Andrew_Palumbo] I worked on the script today and am ready to test my changes but when trying to launch mahout through the shell on azure I kept getting OOM errors, how much memory do you guys typically give the shell in your maven opts?

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854448#comment-15854448 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

Pull request attached here: https://github.com/apache/mahout/pull/279

Let me know thoughts on evolving this and next set of functions to cook up.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854658#comment-15854658 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~Andrew_Palumbo] had a thought here, what constitutes a failure here, do we want to establish thresholds for times where when a test takes longer than the time allocated we declare it a failure, or is failure more complicated like when you run into exceptions etc, thoughts on this?

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884459#comment-15884459 ]

Andrew Palumbo commented on MAHOUT-1904:
----------------------------------------

I would be happy here with an extension of the matrix multiplication timer in {{/bin/examples/}} with an a computational trigger for now.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884474#comment-15884474 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~Andrew_Palumbo] can you take a look at the pull request and add your comments there , how close is the pull request code to what you're asking, do we just need the computational trigger?

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884490#comment-15884490 ]

Andrew Palumbo commented on MAHOUT-1904:
----------------------------------------

What we really need is a set of predetermined parameters for the timer function. just something quick that we can run to test and ensure that everything is working.

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884975#comment-15884975 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

ok lets step back a little bit and think through this, if the goal of this test harness is to test mahout across different hardware configurations the predetermined parameters would be different in my mind across every operation which would mean that we need a set of pre-defined parameters for each/all of our operations, if I look at the initial implementation of timeSparseDRMMul it has the following parameters:
def timeSparseDRMMMul(m: Int, n: Int, s: Int, para: Int, pctDense: Double = .20, seed: Long = 1234L)

If I look at the above the predefined parameters are para,pctDense,seed, is this what you are thinking as well?

So I could see us having a map where the keys are the operations and the values are a list of the predefined parameters, my question then would be this, how does one determine whether an operation worked based on the value of these predefined parameters, doesnt this require thresholds?

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Saikat Kanjilal (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888589#comment-15888589 ]

Saikat Kanjilal commented on MAHOUT-1904:
-----------------------------------------

[~Andrew_Palumbo] care to comment on my observations above, how do you want to proceed on this, would really love to get some basic running and am trying to understand how far away that is from my initial pull request?

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.0
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Musselman (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Musselman updated MAHOUT-1904:
-------------------------------------
Fix Version/s: (was: 0.13.0)
0.13.1

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.1
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Sprint: (was: Jan/Feb-2017)

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.1
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Andrew Palumbo (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Palumbo updated MAHOUT-1904:
-----------------------------------
Sprint: Jan/Feb-2016

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.1
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Trevor Grant (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896770#comment-15896770 ]

Trevor Grant commented on MAHOUT-1904:
--------------------------------------

[~kanjilal] I think you're thinking too hard about this.

Just run the function with some reasonable parameters at the end so we can test whatever by simply `:load examples/bin/SparseSparseDrmTimer.mscala`

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.1
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Trevor Grant (JIRA)]

[ https://issues.apache.org/jira/browse/MAHOUT-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Trevor Grant updated MAHOUT-1904:
---------------------------------
Fix Version/s: (was: 0.13.1)
0.13.2

Post by Andrew Palumbo (JIRA)
Create a test harness to test mahout across different hardware configurations
-----------------------------------------------------------------------------
Key: MAHOUT-1904
URL: https://issues.apache.org/jira/browse/MAHOUT-1904
Project: Mahout
Issue Type: Task
Affects Versions: 0.12.2
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
Priority: Critical
Labels: release, test
Fix For: 0.13.2
Creat a set of simple scala programs to be run as a test harness for Linux amd/intel, mac, and avx2(default).

--
This message was sent by Atlassian JIRA
(v6.4.14#64029)