Big Data Architecture

A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just look at the picture in this articles, it only covers a small fraction of the technologies in the Big Data industry…

Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS & GCP and Data Center infrastructure to answer the basic questions of anyone starting their way in the big data world.

how to transform data (TXT, CSV, TSV, JSON) into Parquet, ORC,AVRO which technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL? GCS? Big Query? Data flow? Data Lab? tensor flow? how to handle streaming? how to manage costs? Performance tips? Security tip? Cloud best practices tips?

In this meetup we shall present lecturers working on several cloud vendors, various big data platforms such hadoop, Data warehourses , startups working on big data products. basically - if it is related to big data - this is THE meetup.

Some of our online materials (mixed content from several cloud vendor):

Website:https://big-data-demystified.ninja

Meetups:

[Big Data Demystified Meetup](https://www.meetup.com/Big-Data-Demystified)

[AWS Big Data Demystified Meetup](https://www.meetup.com/AWS-Big-Data-Demystified/)

Youtube channels:

[Big Data Demystified Youtube channel](https://www.youtube.com/channel/UCMSdNB0fGmX5dXI7S7Y_LFA?view_as=subscriber)

[AWS Big Data Demystified Youtube Channel](https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber)

Facebook Group : https://www.facebook.com/groups/amazon.aws.big.data.demystified/

Facebook page

• Audience: Data Engineers, Data Science, DevOps Engineers ,Big Data Architects , Solution Architects, CTO,VP R&D