Discussion:
[DISCUSS] remove getMahoutHome from sparkbindings
Trevor Grant
2017-06-18 14:04:54 UTC
Permalink
getMahoutHome()

https://github.com/apache/mahout/blob/08e02602e947ff945b9bd73ab5f0b45863df3e53/spark/src/main/scala/org/apache/mahout/sparkbindings/package.scala#L208

seems to not be used anywhere other than the Flink bindings.

I personally don't like the idea of REQUIRING mahout to be fully installed-
and at the moment, at least for Spark- we don't need to. One can simply use
maven coordinates in POM or add them when launching spark
job/shell/zeppelin.

I'm assuming this method existed to support the old spark-shell, but I'd
like to drop it to prevent anyone from introducing new code that relies on
it.

However since I didn't put it there and don't know all use cases, I want to
open up for discussion before I start a JIRA ticket.

thoughts?

tg
Dmitriy Lyubimov
2017-06-21 21:11:34 UTC
Permalink
i think the idea was that mahout has control over application jars and
guarantees that the minimum amount of mahout jars is added to the
application. In order to do that, it needs to be able to figure the home
(even if it is an uber jar, as some devs have been pushing for for some
time now, so it might actually happen)
Post by Trevor Grant
getMahoutHome()
https://github.com/apache/mahout/blob/08e02602e947ff945b9bd73ab5f0b4
5863df3e53/spark/src/main/scala/org/apache/mahout/
sparkbindings/package.scala#L208
seems to not be used anywhere other than the Flink bindings.
I personally don't like the idea of REQUIRING mahout to be fully installed-
and at the moment, at least for Spark- we don't need to. One can simply use
maven coordinates in POM or add them when launching spark
job/shell/zeppelin.
I'm assuming this method existed to support the old spark-shell, but I'd
like to drop it to prevent anyone from introducing new code that relies on
it.
However since I didn't put it there and don't know all use cases, I want to
open up for discussion before I start a JIRA ticket.
thoughts?
tg
Loading...