Spark and Redshift: Which is better for big #data?
Every day, we talk to companies who are in the early phases of building our their data infrastructure. A lot of times these conversations circle around which technology to pick for which job. For example, we often get the question “what’s better — Spark or Amazon Redshift?”, or “which one should we be using?”. Spark and Redshift are two very different technologies. It’s not an either / or, it’s more of a “when do I use what?”. In this post, I’ll lay out some of the differences and when to use which technology.At the time of this post, if you look under the hood of the most advanced tech start-ups in Silicon Valley, you will likely find both Spark and Redshift. Spark is getting a little bit more attention these days because it’s a new shiny toy. But they cover different use cases (“dish washer vs. (...)