DTS 3122 units300 LevelFirst Semester

Big Data Computing

B.Sc. Data Science, University of Uyo

What changes when the data no longer fits on one machine. Distributed storage and computation, the MapReduce model, Hadoop and Spark, and the architectural patterns of modern data platforms.

On Areté, DTS 312 comes with

  • 3 recommended textbooks
  • an AI tutor that has read this course’s outline and notes
Sign in to study DTS 312

Free for University of Uyo students — sign in with your email, no password.

Course outline

  1. 01Characterising big data: volume, velocity, variety, veracity and value
  2. 02Why scale-up fails and scale-out wins; commodity clusters and fault tolerance
  3. 03Distributed file systems: HDFS architecture, blocks, replication, NameNode and DataNodes
  4. 04The MapReduce programming model: map, shuffle, reduce; worked word-count and join examples
  5. 05The Hadoop ecosystem: YARN, Hive, HBase, Sqoop, Flume
  6. 06Apache Spark: RDDs, DataFrames, lazy evaluation, transformations vs actions, in-memory computation
  7. 07Spark SQL and PySpark for data processing
  8. 08Stream processing concepts: Kafka, windowing, and batch vs streaming architectures
  9. 09NoSQL stores and the CAP theorem; cloud big-data services

Recommended textbooks

  • Hadoop: The Definitive Guide — Tom White

    4th ed. — the reference for HDFS and MapReduce

  • Learning Spark — Damji, Wenig, Das & Lee

    2nd ed. — covers the modern DataFrame API; free from Databricks

  • Designing Data-Intensive Applications — Martin Kleppmann

    The best book on distributed data systems; read it slowly over the year

How to pass DTS 312

  • Be able to trace a word-count through map → shuffle → reduce by hand; it is the classic exam question and it teaches the model properly
  • Install Spark locally (or use a free Databricks Community workspace) and run PySpark on a few million rows — the concepts stay abstract until you see a job execute
  • Understand WHY Spark is faster than MapReduce (in-memory, DAG scheduling, lazy evaluation) rather than just asserting it
  • The CAP theorem is examined nearly every year: know what each letter means and why you can only have two

Other 300 Level courses

Lecture notes, practice questions and the AI tutor for DTS 312 are available once you sign in.

Sign in to Areté