Big Data (BIT478CO)
Big data paradigms in business intelligence — MapReduce workflow anatomy, NoSQL databases (HBase, Cassandra, MongoDB), HDFS storage, and Hadoop HiveQL tools.
Unit 1: Introduction to Big Data5 hrs
- Overview of Big Data and comparison with traditional database architectures
- Background of Data Analytics and emergence of distributed systems
- Big Data usage in distributed systems and cloud platforms
- Development history and milestones of Big Data technologies
- Current trends in Big Data Analytics
- Benefits and real-world applications of Big Data (telecom, healthcare, e-commerce, banking)
Unit 2: MapReduce Applications8 hrs
- MapReduce fundamentals and programming model
- MapReduce workflows, mappers, reducers, partitioners, and combiners
- Anatomy of a MapReduce job execution run
- Fault tolerance, node failures, and speculative execution
- Real-world problems solved via MapReduce
- Scalability goals, optimization techniques, and data locality exploitation
- Parallel efficiency and performance bottlenecks of MapReduce
Unit 3: Data Management & Taxonomy of NoSQL Implementations12 hrs
- Structured, semi-structured, and unstructured data management
- Taxonomy of NoSQL implementations: Key-Value, Document, Column-Family, and Graph stores
- Schemaless database designs and CAP theorem implications (Consistency, Availability, Partition tolerance)
- Basic architecture, data models, and query mechanisms of Apache HBase, Apache Cassandra, and MongoDB
- Partitioning, sharding, replication, and composing analytical calculations over NoSQL datastores
Unit 4: Fundamentals of HADOOP10 hrs
- Analyzing data at scale with Apache Hadoop
- Hadoop Distributed File System (HDFS): NameNode, DataNode, Secondary NameNode, blocks, and replication topology
- HDFS command line interface and file operations
- Hadoop Streaming and Hadoop Pipes for multi-language execution
- Hadoop I/O: Data integrity, compression codecs, serialization formats (Avro, Parquet, SequenceFiles)
Unit 5: Hadoop Tools: HBase, Cassandra, Pig, and Hive10 hrs
- Apache HBase architecture: RegionServers, ZooKeeper coordination, and column-family storage
- Apache Cassandra peer-to-peer gossip protocol and CQL operations
- Apache Pig: Architecture, Pig Latin execution environment, data types, and relational operations
- Apache Hive: Hive architecture, HiveQL queries, metastore configurations, managed vs external tables, and partitioning
Reference books
- White, Tom, Hadoop: The Definitive Guide (3rd/4th ed.), O'Reilly Media.
- Minelli, Michael, Michelle Chambers, and Ambiga Dhiraj, Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends for Today's Businesses, Wiley.
- Sadalage, Pramod J. and Martin Fowler, NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence, Addison-Wesley Professional.
- Sammer, Eric, Hadoop Operations, O'Reilly Media.
- George, Lars, HBase: The Definitive Guide, O'Reilly Media.