sparkbizapps

Please post your queries here

Waiting for your replies/comments

realtime is big benefit

Waiting for your replies/comments

in the moment analytics

Spark is part of the Apache open source ecosystem - so, anyone can commit updates to it. Hence, any improvements are shared with the whole ODS community

There is huge value in an open source analytics platform - (1) portability of solutions

Spark is the predominant open-source modeling tools for data science: machine learning, streaming analytics, graph analysis.

Tell us your view on the role of #ApacheSpark in DataScience

Another value: easier collaboration (common framework)

Data Science uses Apache Spark to perform investigative analytics. Spark provides an interactive statistical environments to perform ad-hoc, exploratory analytics in order to gain insights and answer questions.

Spark is, as Kirk states, part of the open Apache ecosystem of code for data science, big data, cognitive computing, etc,, including Hadoop, Kafka, and others.

Rapid prototyping!

Quick analytics helps you fail faster / learn quicker.

Spark will emerge as the "operating system" for #analytics. Analytics needs parallelism for speed. Spark provides the basic infrastructure. #sparkbizapps

Here's my recent IBM Big Data & Analytics Hub blog on Spark and R in the open analytics ecosystem. http://www.ibmbigdat...

spark enables data science to be relevant around what's happening now

Open source is critical, especially in the world of data science. Spark offers real collaboration for everyone.

Jim K. is right on! Spark has great components: http://www.slideshar... http://image.slidesh...

#Spark is beginning to understand what the #IBMz #mainframe has known for decades - resource management is critical to performance. #sparkbizapps

@furrier Hi John! Great to see you.

@furrier Great to see you John. #sparkbizapps

Spark is the common development tool around which open data science communities can converge. Its open-source ubiquitious, and real-time/in-memory/interactive strengths, make it well-suited to embedded IoT analytics and other key apps.

Spark in particular is making it easier to do the analysis w/o having to think about the mechanics of Map/Reduce

Some folks think Spark is in competition with Hadoop. Not true! See my "Lord of the Things" article: https://www.mapr.com...

As Apache Spark evolves via the Apache eco-system so does the toolset for Data Science. Data Science evolves as well innovating the statistical models produced.

@DrZeydy is exactly right -- Spark looks like a fighter jet next to Map/Reduce (a horseless carriage) https://www.mapr.com...

Time for Question #3 on the top of your screen

@TamaraMcCleary Here's my recent blog on collaboration and workflow in open data science initiatives involving Spark and R. http://www.ibmbigdat...

.@IBMbigdata Unlike other tools in the Hadoop family, which tend to focus on technology (i.e. queuing messages), Apache Spark is an important tool for open data science becasue data scientists can leverage HDFS, which is growing in popularity

At @BoozAllen, we are focusing on Agile Analytics, Fast Iterative Prototyping, MVP (Minimum Viable Product), etc. = #DataOps ...http://www.oreilly.c...

@craigbrownphd I forgot to say "welcome," Craig. I always enjoy your contributions.

Please post your replies here

Gentle reminder to post your replies here

Waiting to hear from you on, Q3

Important components of Open #DataScience include the Apache ecosystem (Spark, Hadoop, Drill, etc.) plus R, Python, Jupyter, and microservices & containers (Docker...)

Most important open-source DS language is R (with Python & Scala not far behind). Most important modeling tool is Spark. Most important open data refinemt platf: Hadoop. Most important open distrib pub/sub commit log platform: Kafka

All sorts of cool tools from Apache: Spark, Accumulo, Kafka, Drill, and Streaming Analytics = http://www.cakesolut...

Spark, R and Hadoop are essential. Most of the other components just enhance what can be delivered by #DataScience

Let us keep this great conversation going! .. Please chime in with your replies

Most important open-source machine-learning libraries include SystemML, TensorFlow, and Mahout.

there should be some security components in the "stack"

Machine Leaning, to Data Science, is like the icing on the cake. You get a functionality out of the statistical model, bringing it to life

All this data dramatically expands the security threat - increases the stakes around IP

Excellent point @jameskobielus ! -- the various open source #MachineLearning libraries are great! http://bit.ly/1PMY20... #abdsc

@dvellante Hello Dave!! ... great to see so many great folks here!

"best" depends on your application. #R is my favorite.

Time for question number 4 : Q#4

@KirkDBorne ditto Kirk - nice gathering of data zealots!

The most important components will depend on the role of the data scientist. A data scientist working on product development will find python more valuable than R and APIs more useful than relational databases.

Check out these open #DeepLearning Libraries for #MachineLearning: http://bit.ly/1JZbTK... #abdsc

Most important open-data sources? Hard to say, but here's an excellent recent list that Bernard Marr compiled: http://www.forbes.co...

@dvellante Absolutely agree Dave. #Cybersecurity is critical w/ the exponential growth of #BigData #Cloud & #IoT devices. #sparkbizapps

@8pathsolutions Right!! Python for operational development (#DataProducts) and R for exploration, discovery, validation,...

Here is my big list of #OpenData repositories, benefits, and characteristics: http://rocketdatasci...

@dvellante I totally agree! Having all of this data is great, but we are so vulnerable now! It is a bit frightening when you think about it in depth.