Mahout Interview Questions and Answers

1.What Is Apache Mahout?

Ans:Apache™ Mahout is a library of scalable machine-learning algorithms, implemented on top of Apache Hadoop® and using the MapReduce paradigm. Machine learning is a discipline of artificial intelligence focused on enabling machines to learn without being explicitly programmed, and it is commonly used to improve future performance based on previous outcomes.

Once big data is stored on the Hadoop Distributed File System (HDFS), Mahout provides the data science tools to automatically find meaningful patterns in those big data sets. The Apache Mahout project aims to make it faster and easier to turn big data into big information.

2.What does Apache Mahout do?

Ans: Mahout supports four main data science use cases:

  • Collaborative filtering – mines user behavior and makes product recommendations (e.g. Amazon recommendations)
  • Clustering – takes items in a particular class (such as web pages or newspaper articles) and organizes them into naturally occurring groups, such that items belonging to the same group are similar to each other
  • Classification – learns from existing categorizations and then assigns unclassified items to the best category
  • Frequent item-set mining – analyzes items in a group (e.g. items in a shopping cart or terms in a query session) and then identifies which items typically appear together.

3.What Is The History Of Apache Mahout? When Did It Start?

Ans: The Mahout project was started by several people involved in the Apache Lucene (open source search) community with an active interest in machine learning and a desire for robust, well-documented, scalable implementations of common machine-learning algorithms for clustering and categorization. The community was initially driven by Ng et al.’s paper “Map-Reduce for Machine Learning on Multicore” (see Resources) but has since evolved to cover much broader machine-learning approaches. Mahout also aims to:

  • Build and support a community of users and contributors such that the code outlives any particular contributor’s involvement or any particular company or university’s funding.
  • Focus on real-world, practical use cases as opposed to bleeding-edge research or unproven techniques.
  • Provide quality documentation and examples.

4.What motivated you to work on Apache Mahout? How do you compare Mahout with Spark and H2O?

Ans: Well, some good friends asked me to answer some questions. From there it was a down-hill slope. First a few questions to be answered. Then some code to be reviewed. Then a few implementations. Suddenly I was a committer and was strong committed to the project.
With respect to Spark and H2O, it is difficult to make direct comparisons. Mahout was many years ahead of these other systems and thus had to commit early on to much more primitive forms of scalable computing in order to succeed. That commitment has lately changed and the new generation of Mahout code supports both Spark and H2O as computational back-ends for modern work.

That inter-relationship makes direct comparison even harder in some ways. I think that there is so much to work on in machine learning that it is hard to say that one project is directly competitive with another when, in fact, they actually work together in many ways.
Clearly Mahout has a huge lead over the other systems in the way that it compiles linear algebra expressions into efficient programs for back-ends like Spark (or H2O). Clearly also, H2O has a huge lead over Spark’s MLLib in terms of numerical performance and sophisticated learning algorithms. Mahout is also the only system that fully supports indicator-based recommendation systems, which is a huge difference as well.

5.What is the Roadmap for Apache Mahout version 1.0?

Ans: The next major version, Mahout 1.0, will contain major changes to the underlying architecture of Mahout, including:

  • Scala:In addition to Java, Mahout users will be able to write jobs using the Scala programming language. Scala makes programming math-intensive applications much easier as compared to Java, so developers will be much more effective.
  • Spark & h2o:Mahout 0.9 and below relied on MapReduce as an execution engine. With Mahout 1.0, users can choose to run jobs either on Spark or h2o, resulting in a significant performance increase.

6.What are the features of Apache Mahout?

Ans: Although relatively young in open source terms, Mahout already has a large amount of functionality, especially in relation to clustering and CF. Mahout’s primary features are:

  • Taste CF. Taste is an open source project for CF started by Sean Owen on SourceForge and donated to Mahout in 2008.
  • Several Mapreduce enabled clustering implementations, including k-Means, fuzzy k-Means, Canopy, Dirichlet, and Mean-Shift.
  • Distributed Naive Bayes and Complementary Naive Bayes classification implementations.
  • Distributed fitness function capabilities for evolutionary programming.
  • Matrix and vector libraries.
  • Examples of all of the above algorithms.

7.Mention some machine learning algorithms exposed by Mahout?

Below is a current list of machine learning algorithms exposed by Mahout.

  • Collaborative Filtering
    • Item-based Collaborative Filtering
    • Matrix Factorization with Alternating Least Squares
    • Matrix Factorization with Alternating Least Squares on Implicit Feedback
  • Classification
    • Naive Bayes
    • Complementary Naive Bayes
    • Random Forest
  • Clustering
    • Canopy Clustering
    • k-Means Clustering
    • Fuzzy k-Means
    • Streaming k-Means
    • Spectral Clustering
  • Dimensionality Reduction
    • Lanczos Algorithm
    • Stochastic SVD
    • Principal Component Analysis
  • Topic Models
    • Latent Dirichlet Allocation
  • Miscellaneous
    • Frequent Pattern Matching
    • RowSimilarityJob
    • ConcatMatrices
    • Colocations

8.What Are The Different Clustering In Mahout?

Ans :Mahout supports several clustering-algorithm implementations, all written in Map-Reduce, each with its own set of goals and criteria:

Canopy: A fast clustering algorithm often used to create initial seeds for other clustering algorithms.

k-Means (and fuzzy k-Means): Clusters items into k clusters based on the distance the items are from the centroid, or center, of the previous iteration.

Mean-Shift: Algorithm that does not require any a priori knowledge about the number of clusters and can produce arbitrarily shaped clusters.

Dirichlet: Clusters based on the mixing of many probabilistic models giving it the advantage

9.Mention Some Use Cases Of Apache Mahout?

Ans :Commercial Use 

  • Adobe AMP uses Mahout’s clustering algorithms to increase video consumption by better user targeting.
  • Accenture uses Mahout as typical example for their Hadoop Deployment Comparison Study
  • AOL use Mahout for shopping recommendations. See slide deck
  • Booz Allen Hamilton uses Mahout’s clustering algorithms. See slide deck
  • Buzzlogic uses Mahout’s clustering algorithms to improve ad targeting
  • uses modified Mahout algorithms for content recommendations
  • DataMine Lab uses Mahout’s recommendation and clustering algorithms to improve our clients’ ad targeting.
  • Drupal users Mahout to provide open source content recommendation solutions.
  • Evolv uses Mahout for its Workforce Predictive Analytics platform.
  • Foursquare uses Mahout for its recommendation engine.
  • Idealo uses Mahout’s recommendation engine.
  • InfoGlutton uses Mahout’s clustering and classification for various consulting projects.
  • Intel ships Mahout as part of their Distribution for Apache Hadoop Software.
  • Intela has implementations of Mahout’s recommendation algorithms to select new offers to send tu customers, as well as to recommend potential customers to current offers. We are also working on enhancing our offer categories by using the clustering algorithms.
  • iOffer uses Mahout’s Frequent Pattern Mining and Collaborative Filtering to recommend items to users.
  • Kauli , one of Japanese Ad network, uses Mahout’s clustering to handle click stream data for predicting audience’s interests and intents.
  • Linked.In Historically, we have used R for model training. We have recently started experimenting with Mahout for model training and are excited about it – also see Hadoop World slides .
  • LucidWorks Big Data uses Mahout for clustering, duplicate document detection, phrase extraction and classification.
  • Mendeley uses Mahout to power Mendeley Suggest, a research article recommendation service.
  • Mippin uses Mahout’s collaborative filtering engine to recommend news feeds
  • Mobage uses Mahout in their analysis pipeline
  • Myrrix is a recommender system product built on Mahout.
  • NewsCred uses Mahout to generate clusters of news articles and to surface the important stories of the day
  • Next Glass uses Mahout
  • Predixion Software uses Mahout’s algorithms to build predictive models on big data
  • Radoop provides a drag-n-drop interface for big data analytics, including Mahout clustering and classification algorithms
  • ResearchGate, the professional network for scientists and researchers, uses Mahout’s recommendation algorithms.
  • Sematext uses Mahout for its recommendation engine
  • uses Mahout’s collaborative filtering engine to recommend member profiles
  • Twitter uses Mahout’s LDA implementation for user interest modeling
  • Yahoo! Mail uses Mahout’s Frequent Pattern Set Mining.
  • 365Media uses Mahout’s Classification and Collaborative Filtering algorithms in its Real-time system named UPTIME and 365Media/Social. 

Academic Use

  • Dicode project uses Mahout’s clustering and classification algorithms on top of HBase.
  • The course Large Scale Data Analysis and Data Mining at TU Berlin uses Mahout to teach students about the parallelization of data mining problems with Hadoop and Mapreduce
  • Mahout is used at Carnegie Mellon University, as a comparable platform to GraphLab
  • The ROBUST project , co-funded by the European Commission, employs Mahout in the large scale analysis of online community data.
  • Mahout is used for research and data processing at Nagoya Institute of Technology , in the context of a large-scale citizen participation platform project, funded by the Ministry of Interior of Japan.
  • Several researches within Digital Enterprise Research Institute NUI Galway use Mahout for e.g. topic mining and modeling of large corpora.
  • Mahout is used in the NoTube EU project.

10.How is it different from doing machine learning in R or SAS?

Ans: Unless you are highly proficient in Java, the coding itself is a big overhead. There’s no way around it, if you don’t know it already you are going to need to learn Java and it’s not a language that flows! For R users who are used to seeing their thoughts realized immediately the endless declaration and initialization of objects is going to seem like a drag. For that reason I would recommend sticking with R for any kind of data exploration or prototyping and switching to Mahout as you get closer to production.

11.What square measure the options of Apache Mahout?

Ans: Although comparatively young in open supply terms, driver already encompasses a great deal of practicality, particularly in relevance bunch and CF. Mahout’s primary options are:

Taste CF. style is associate open supply project for CF started by Sean Owen on SourceForge and given to driver in 2008.

Several Mapreduce enabled bunch implementations, as well as k-Means, fuzzy k-Means, Canopy, Dirichlet, and Mean-Shift.

Distributed Naive Thomas {bayes|mathematician} and Complementary Naive Bayes classification implementations.