I wrote a blog post about Boto2 and EMR clusters a few months ago. Today I’m going to show how to create EMR clusters using Boto3. Boto3 documentation is available at here.
nn
import boto3nn__author__ = 'dedunu'nnconnection = boto3.client(n 'emr',n region_name='us-west-1',n aws_access_key_id='<Your AWS Access Key>',n aws_secret_access_key='<You AWS Secred Key>',n)nncluster_id = connection.run_job_flow(n Name='test_emr_job_with_boto3',n LogUri='s3://<your s3 location>',n ReleaseLabel='emr-4.2.0',n Instances={n 'InstanceGroups': [n {n 'Name': "Master nodes",n 'Market': 'ON_DEMAND',n 'InstanceRole': 'MASTER',n 'InstanceType': 'm1.large',n 'InstanceCount': 1,n },n {n 'Name': "Slave nodes",n 'Market': 'ON_DEMAND',n 'InstanceRole': 'CORE',n 'InstanceType': 'm1.large',n 'InstanceCount': 2,n }n ],n 'Ec2KeyName': '<Ec2 Keyname>',n 'KeepJobFlowAliveWhenNoSteps': True,n 'TerminationProtected': False,n 'Ec2SubnetId': '<Your Subnet ID>',n },n Steps=[],n VisibleToAllUsers=True,n JobFlowRole='EMR_EC2_DefaultRole',n ServiceRole='EMR_DefaultRole',n Tags=[n {n 'Key': 'tag_name_1',n 'Value': 'tab_value_1',n },n {n 'Key': 'tag_name_2',n 'Value': 'tag_value_2',n },n ],n)nnprint (cluster_id['JobFlowId'])n
nn
Gists
nn
nn
Tags
nn
- n
- emr
- amazon
- boto3
- boto
- aws
n
n
n
n
n
n