Until now, design patterns for the MapReduce framework have been scattered among various research papers, blogs, and books. This handy guide brings together a unique collection of valuable MapReduce patterns that will save you time and effort regardless of the domain, language, or development framework you’re using.
Each pattern is explained in context, with pitfalls and caveats clearly identified to help you avoid common design mistakes when modeling your big data architecture. This book also provides a complete overview of MapReduce that explains its origins and implementations, and why design patterns are so important. All code examples are written for Hadoop.
Summarization patterns: get a top-level view by summarizing and grouping data
Filtering patterns: view data subsets such as records generated from one user
Data organization patterns: reorganize data to work with other systems, or to make MapReduce analysis easier
Join patterns: analyze different datasets together to discover interesting relationships
Metapatterns: piece together several patterns to solve multi-stage problems, or to perform several analytics in the same job
Input and output patterns: customize the way you use Hadoop to load or store data
"A clear exposition of MapReduce programs for common data processing patterns—this book is indespensible for anyone using Hadoop."
--Tom White, author of Hadoop: The Definitive Guide
**
Book Description
Building Effective Algorithms and Analytics for Hadoop and Other Systems
About the Author
Chief Technology Officer, ClearEdge IT Solutions
Donald Miner is an avid user of Apache Hadoop and a practitioner of data science. He serves as Chief Technology Officer at ClearEdge IT Solutions, a company that provides Big Data professional services. He is author of the O'Reilly book MapReduce Design Patterns, which is based on his experiences as a MapReduce developer. Donald has architected and implemented a number of mission-critical and large-scale Hadoop systems within the U.S. Government and Fortune 500 companies. He received his PhD from the University of Maryland, Baltimore County in Computer Science, where he focused on Machine Learning and Multi-Agent Systems. He lives in Maryland with his wife and two young sons.Twitter: @donaldpminer
Adam Shook is a Software Engineer at ClearEdge IT Solutions, LLC,working with a number of big data technologies such as Hadoop, Accumulo, Pig, and ZooKeeper. Shook graduated with a B.S. in Computer Science from the University of Maryland Baltimore County (UMBC) and took a job building a new high-performance graphics engine for a game studio. Seeking new challenges, he enrolled in the graduate program at UMBC with a focus on distributed computing technologies. He quickly found development work as a U.S. government contractor on a large-scale Hadoop deployment. Shook is involved in developing and instructing training curriculum for both Hadoop and Pig. He spends what little free time he has working on side projects and playing video games.
Description:
Until now, design patterns for the MapReduce framework have been scattered among various research papers, blogs, and books. This handy guide brings together a unique collection of valuable MapReduce patterns that will save you time and effort regardless of the domain, language, or development framework you’re using.
Each pattern is explained in context, with pitfalls and caveats clearly identified to help you avoid common design mistakes when modeling your big data architecture. This book also provides a complete overview of MapReduce that explains its origins and implementations, and why design patterns are so important. All code examples are written for Hadoop.
"A clear exposition of MapReduce programs for common data processing patterns—this book is indespensible for anyone using Hadoop."
--Tom White, author of Hadoop: The Definitive Guide
**
Book Description
Building Effective Algorithms and Analytics for Hadoop and Other Systems
About the Author
Chief Technology Officer, ClearEdge IT Solutions
Donald Miner is an avid user of Apache Hadoop and a practitioner of data science. He serves as Chief Technology Officer at ClearEdge IT Solutions, a company that provides Big Data professional services. He is author of the O'Reilly book MapReduce Design Patterns, which is based on his experiences as a MapReduce developer. Donald has architected and implemented a number of mission-critical and large-scale Hadoop systems within the U.S. Government and Fortune 500 companies. He received his PhD from the University of Maryland, Baltimore County in Computer Science, where he focused on Machine Learning and Multi-Agent Systems. He lives in Maryland with his wife and two young sons.Twitter: @donaldpminer
Adam Shook is a Software Engineer at ClearEdge IT Solutions, LLC,working with a number of big data technologies such as Hadoop, Accumulo, Pig, and ZooKeeper. Shook graduated with a B.S. in Computer Science from the University of Maryland Baltimore County (UMBC) and took a job building a new high-performance graphics engine for a game studio. Seeking new challenges, he enrolled in the graduate program at UMBC with a focus on distributed computing technologies. He quickly found development work as a U.S. government contractor on a large-scale Hadoop deployment. Shook is involved in developing and instructing training curriculum for both Hadoop and Pig. He spends what little free time he has working on side projects and playing video games.