Peter Skomoroch, profile picture
Uploaded byPeter Skomoroch
PDF, PPTX1,440 views

Elasticwulf Pycon Talk

This document discusses using Python and Amazon EC2 for parallel programming and clustering. It introduces ElasticWulf, which provides Amazon Machine Images preconfigured for clustering. It also covers MPI (message passing interface) basics in Python, including broadcasting, scattering, gathering, and reducing data across nodes. A demo is given of launching an ElasticWulf cluster on EC2, configuring it for MPI, and running a simple parallel pi calculation example using mpi4py.

Download as PDF, PPTX
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk
Elasticwulf Pycon Talk

More Related Content

PPTX
Airflow - a data flow engine
byWalter Liu
 
PDF
Deep Learning을 위한 AWS 기반 인공 지능(AI) 서비스 (윤석찬)
byAmazon Web Services Korea
 
PPT
Hw09 Building Data Intensive Apps A Closer Look At Trending Topics.Org
byCloudera, Inc.
 
PPTX
Google colab installing ml libraries
bySaravanakumar viswanathan
 
PPTX
Enable IPv6 on Route53 AWS ELB, docker and node App
byFyllo
 
PPT
Hw09 Hadoop Applications At Yahoo!
byCloudera, Inc.
 
PDF
Prototyping Data Intensive Apps: TrendingTopics.org
byPeter Skomoroch
 
PDF
Summingbird: Streaming Portable, MapReduce
byDataWorks Summit
 
Airflow - a data flow engine
byWalter Liu
 
Deep Learning을 위한 AWS 기반 인공 지능(AI) 서비스 (윤석찬)
byAmazon Web Services Korea
 
Hw09 Building Data Intensive Apps A Closer Look At Trending Topics.Org
byCloudera, Inc.
 
Google colab installing ml libraries
bySaravanakumar viswanathan
 
Enable IPv6 on Route53 AWS ELB, docker and node App
byFyllo
 
Hw09 Hadoop Applications At Yahoo!
byCloudera, Inc.
 
Prototyping Data Intensive Apps: TrendingTopics.org
byPeter Skomoroch
 
Summingbird: Streaming Portable, MapReduce
byDataWorks Summit
 

What's hot

PDF
Apache spark presentation
byMahboob Hussain
 
PPTX
Terraform on Azure
byJulien Corioland
 
PDF
Pytorch Ecosystem Day: The 珞 Ecosystem
byLysandre Debut
 
PDF
SciPy 2019: How to Accelerate an Existing Codebase with Numba
bystan_seibert
 
PPS
Storm presentation
byShyam Raj
 
PDF
Introduction to Apache Storm - Concept & Example
byDung Ngua
 
PDF
Intro to Airflow: Goodbye Cron, Welcome scheduled workflow management
byBurasakorn Sabyeying
 
PDF
Luigi presentation NYC Data Science
byErik Bernhardsson
 
PPTX
Slide #1:Introduction to Apache Storm
byMd. Shamsur Rahim
 
PDF
Managing data workflows with Luigi
byTeemu Kurppa
 
PPTX
Community-Driven Graphs with JanusGraph
byJason Plurad
 
PPTX
Custom object detection_gcp
byVaibhav Sahu
 
PPTX
Paris DataGeek - SummingBird
bySofian Djamaa
 
PDF
Big Data Berlin - Criteo
bySofian Djamaa
 
PPTX
Apache Storm Internals
byHumoyun Ahmedov
 
PDF
Building infrastructure as code using Terraform - DevOps Krakow
byAnton Babenko
 
PPTX
Tensorflow vs MxNet
byAshish Bansal
 
PDF
Luigi future
byErik Bernhardsson
 
PDF
Luigi presentation OA Summit
byOpen Analytics
 
Apache spark presentation
byMahboob Hussain
 
Terraform on Azure
byJulien Corioland
 
Pytorch Ecosystem Day: The 珞 Ecosystem
byLysandre Debut
 
SciPy 2019: How to Accelerate an Existing Codebase with Numba
bystan_seibert
 
Storm presentation
byShyam Raj
 
Introduction to Apache Storm - Concept & Example
byDung Ngua
 
Intro to Airflow: Goodbye Cron, Welcome scheduled workflow management
byBurasakorn Sabyeying
 
Luigi presentation NYC Data Science
byErik Bernhardsson
 
Slide #1:Introduction to Apache Storm
byMd. Shamsur Rahim
 
Managing data workflows with Luigi
byTeemu Kurppa
 
Community-Driven Graphs with JanusGraph
byJason Plurad
 
Custom object detection_gcp
byVaibhav Sahu
 
Paris DataGeek - SummingBird
bySofian Djamaa
 
Big Data Berlin - Criteo
bySofian Djamaa
 
Apache Storm Internals
byHumoyun Ahmedov
 
Building infrastructure as code using Terraform - DevOps Krakow
byAnton Babenko
 
Tensorflow vs MxNet
byAshish Bansal
 
Luigi future
byErik Bernhardsson
 
Luigi presentation OA Summit
byOpen Analytics
 

Similar to Elasticwulf Pycon Talk

PPTX
The Message Passing Interface (MPI) in Layman's Terms
byJeff Squyres
 
PPTX
Role of python in hpc
byDr Reeja S R
 
PDF
mpi4py.pdf
byA Jorge Garcia
 
PDF
Integrating Existing C++ Libraries into PySpark with Esther Kundin
byDatabricks
 
PDF
AWS Community Day Bangkok 2019 - How AWS Parallel Cluster can accelerate high...
byAWS User Group - Thailand
 
PDF
Systems Bioinformatics Workshop Keynote
byDeepak Singh
 
PDF
Parallel Processing with IPython
byEnthought, Inc.
 
PDF
Microservices and Teraflops: Effortlessly Scaling Data Science with PyWren wi...
byDatabricks
 
PDF
More mpi4py
byA Jorge Garcia
 
PDF
Mpi4py
byA Jorge Garcia
 
ODP
Parallel Programming on the ANDC cluster
bySudhang Shankar
 
PPT
AWS (Hadoop) Meetup 30.04.09
byChris Purrington
 
PDF
Build and deploy scientific Python Applications
byRamakrishna Reddy
 
PPTX
2020 re:Cap
byCobus Bernard
 
PDF
HPC on AWS
byReadMaloney
 
KEY
R Jobs on the Cloud
byJohn Doxaras
 
PDF
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
byJason Liu
 
PDF
Data Science
bySubhajit75
 
PPTX
Designing HPC, Deep Learning, and Cloud Middleware for Exascale Systems
byinside-BigData.com
 
PDF
magellan_mongodb_workload_analysis
byPraveen Narayanan
 
The Message Passing Interface (MPI) in Layman's Terms
byJeff Squyres
 
Role of python in hpc
byDr Reeja S R
 
mpi4py.pdf
byA Jorge Garcia
 
Integrating Existing C++ Libraries into PySpark with Esther Kundin
byDatabricks
 
AWS Community Day Bangkok 2019 - How AWS Parallel Cluster can accelerate high...
byAWS User Group - Thailand
 
Systems Bioinformatics Workshop Keynote
byDeepak Singh
 
Parallel Processing with IPython
byEnthought, Inc.
 
Microservices and Teraflops: Effortlessly Scaling Data Science with PyWren wi...
byDatabricks
 
More mpi4py
byA Jorge Garcia
 
Mpi4py
byA Jorge Garcia
 
Parallel Programming on the ANDC cluster
bySudhang Shankar
 
AWS (Hadoop) Meetup 30.04.09
byChris Purrington
 
Build and deploy scientific Python Applications
byRamakrishna Reddy
 
2020 re:Cap
byCobus Bernard
 
HPC on AWS
byReadMaloney
 
R Jobs on the Cloud
byJohn Doxaras
 
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
byJason Liu
 
Data Science
bySubhajit75
 
Designing HPC, Deep Learning, and Cloud Middleware for Exascale Systems
byinside-BigData.com
 
magellan_mongodb_workload_analysis
byPraveen Narayanan
 

More from Peter Skomoroch

PPT
O'Reilly Strata: Distilling Data Exhaust
byPeter Skomoroch
 
PDF
Product Management for AI
byPeter Skomoroch
 
PPTX
Building Competitive Moats With Data
byPeter Skomoroch
 
PPTX
SF Data Science: Developing Data Products
byPeter Skomoroch
 
PPTX
Developing Data Products
byPeter Skomoroch
 
PDF
Street Fighting Data Science
byPeter Skomoroch
 
PDF
Managing Machines: The New AI Dev Stack
byPeter Skomoroch
 
KEY
Geo Analytics Tutorial - Where 2.0 2011
byPeter Skomoroch
 
PDF
Executive Briefing: Why managing machines is harder than you think
byPeter Skomoroch
 
PDF
Practical Problem Solving with Data - Onlab Data Conference, Tokyo
byPeter Skomoroch
 
PDF
Rapid Data Exploration With Hadoop
byPeter Skomoroch
 
PPTX
Bridging the AI Gap: Building Stakeholder Support
byPeter Skomoroch
 
PPTX
Skills, Reputation, and Search
byPeter Skomoroch
 
PDF
Data Mashups -Data Science Summit
byPeter Skomoroch
 
PPTX
LinkedIn Endorsements: Reputation, Virality, and Social Tagging
byPeter Skomoroch
 
O'Reilly Strata: Distilling Data Exhaust
byPeter Skomoroch
 
Product Management for AI
byPeter Skomoroch
 
Building Competitive Moats With Data
byPeter Skomoroch
 
SF Data Science: Developing Data Products
byPeter Skomoroch
 
Developing Data Products
byPeter Skomoroch
 
Street Fighting Data Science
byPeter Skomoroch
 
Managing Machines: The New AI Dev Stack
byPeter Skomoroch
 
Geo Analytics Tutorial - Where 2.0 2011
byPeter Skomoroch
 
Executive Briefing: Why managing machines is harder than you think
byPeter Skomoroch
 
Practical Problem Solving with Data - Onlab Data Conference, Tokyo
byPeter Skomoroch
 
Rapid Data Exploration With Hadoop
byPeter Skomoroch
 
Bridging the AI Gap: Building Stakeholder Support
byPeter Skomoroch
 
Skills, Reputation, and Search
byPeter Skomoroch
 
Data Mashups -Data Science Summit
byPeter Skomoroch
 
LinkedIn Endorsements: Reputation, Virality, and Social Tagging
byPeter Skomoroch
 

Recently uploaded

PPTX
xCapture v3: Efficient, Always-On Thread Level Observability with eBPF by Tan...
byScyllaDB
 
PDF
Immer vorne mit dabei sein - Roaming leicht gemacht
bypanagenda
 
PPTX
ScalaF: Functional Refactoring and Automated Code Transformations
byhelloshrikha
 
PPTX
Session 6 Specialized AI Associate Series: The GenAI Experience in UiPath Doc...
byDianaGray10
 
PDF
Top 11 Sites to Buy Verified GitHub Accounts in 2025
byBuy GitHub Accounts Looking for a verified GitHub account?
 
PPTX
Mosaic: The Agentic AI Video Editing Platform
byDaniel Gimness
 
PPTX
Autonomous Convoy Routing via Drone Swarms and Multi-Modal Threat Detection
byIvan Ruchkin
 
PDF
FME Realize in the Real World - The Power of Spatial Computing in Action
bySafe Software
 
PDF
Top-10-Cloud-Service-Companies-in-2025-Market-Leaders-and-Innovators.pdf
byArtjoker Software Development Company
 
PDF
RaBitQ, JSON Index, Schema Evolution, What's New in Milvus 2.6
byZilliz
 
PDF
Energia Nowa and byteLAKE Partner to Develop Intelligent Services for Energy ...
bybyteLAKE
 
PDF
The End of the Missed Call - How AI Recaptures Lost Leads and Secures ROI
byiovox
 
PPTX
Artificial Intelligence lecture slides.pptx
bymaimelabernard6
 
PDF
Generation Data Sets - Unique, Useful... Understandable?.pdf
byPrecisely
 
PDF
Morgenbooster: Navigating Ai Criticism with Systems Thinking
by1508 A/S
 
PDF
Memo No.3006141 dt.25.10.2025- SOP for Grant of Online Change Of Land Use
byRaghu Ram
 
PDF
Transcript: Embedding sustainability: Tips for ebook and print production - T...
byBookNet Canada
 
PDF
The State of Adventure Capital - Grant Gregory | Cantos
byRazin Mustafiz
 
PDF
G.O.RT.No. 1173_LRS Extension of time limit
byRAGHU RAM
 
PDF
Ready, set, go: Pre-fall sales trends and data-driven insights - Tech Forum 2025
byBookNet Canada
 
xCapture v3: Efficient, Always-On Thread Level Observability with eBPF by Tan...
byScyllaDB
 
Immer vorne mit dabei sein - Roaming leicht gemacht
bypanagenda
 
ScalaF: Functional Refactoring and Automated Code Transformations
byhelloshrikha
 
Session 6 Specialized AI Associate Series: The GenAI Experience in UiPath Doc...
byDianaGray10
 
Top 11 Sites to Buy Verified GitHub Accounts in 2025
byBuy GitHub Accounts Looking for a verified GitHub account?
 
Mosaic: The Agentic AI Video Editing Platform
byDaniel Gimness
 
Autonomous Convoy Routing via Drone Swarms and Multi-Modal Threat Detection
byIvan Ruchkin
 
FME Realize in the Real World - The Power of Spatial Computing in Action
bySafe Software
 
Top-10-Cloud-Service-Companies-in-2025-Market-Leaders-and-Innovators.pdf
byArtjoker Software Development Company
 
RaBitQ, JSON Index, Schema Evolution, What's New in Milvus 2.6
byZilliz
 
Energia Nowa and byteLAKE Partner to Develop Intelligent Services for Energy ...
bybyteLAKE
 
The End of the Missed Call - How AI Recaptures Lost Leads and Secures ROI
byiovox
 
Artificial Intelligence lecture slides.pptx
bymaimelabernard6
 
Generation Data Sets - Unique, Useful... Understandable?.pdf
byPrecisely
 
Morgenbooster: Navigating Ai Criticism with Systems Thinking
by1508 A/S
 
Memo No.3006141 dt.25.10.2025- SOP for Grant of Online Change Of Land Use
byRaghu Ram
 
Transcript: Embedding sustainability: Tips for ebook and print production - T...
byBookNet Canada
 
The State of Adventure Capital - Grant Gregory | Cantos
byRazin Mustafiz
 
G.O.RT.No. 1173_LRS Extension of time limit
byRAGHU RAM
 
Ready, set, go: Pre-fall sales trends and data-driven insights - Tech Forum 2025
byBookNet Canada