How to create an EMR cluster using Boto3?

I wrote a blog post about Boto2 and EMR clusters a few months ago. Today I’m going to show how to create EMR clusters using Boto3. Boto3 documentation is available at here.

nn

import boto3nn__author__ = 'dedunu'nnconnection = boto3.client(n    'emr',n    region_name='us-west-1',n    aws_access_key_id='<Your AWS Access Key>',n    aws_secret_access_key='<You AWS Secred Key>',n)nncluster_id = connection.run_job_flow(n    Name='test_emr_job_with_boto3',n    LogUri='s3://<your s3 location>',n    ReleaseLabel='emr-4.2.0',n    Instances={n        'InstanceGroups': [n            {n                'Name': "Master nodes",n                'Market': 'ON_DEMAND',n                'InstanceRole': 'MASTER',n                'InstanceType': 'm1.large',n                'InstanceCount': 1,n            },n            {n                'Name': "Slave nodes",n                'Market': 'ON_DEMAND',n                'InstanceRole': 'CORE',n                'InstanceType': 'm1.large',n                'InstanceCount': 2,n            }n        ],n        'Ec2KeyName': '<Ec2 Keyname>',n        'KeepJobFlowAliveWhenNoSteps': True,n        'TerminationProtected': False,n        'Ec2SubnetId': '<Your Subnet ID>',n    },n    Steps=[],n    VisibleToAllUsers=True,n    JobFlowRole='EMR_EC2_DefaultRole',n    ServiceRole='EMR_DefaultRole',n    Tags=[n        {n            'Key': 'tag_name_1',n            'Value': 'tab_value_1',n        },n        {n            'Key': 'tag_name_2',n            'Value': 'tag_value_2',n        },n    ],n)nnprint (cluster_id['JobFlowId'])n

nn

Gists

nn

nn

Tags

nn

    n

  • emr
  • n

  • amazon
  • n

  • boto3
  • n

  • boto
  • n

  • aws
  • n

n

Proudly powered by WordPress

Discover more from Dedunu

Subscribe now to keep reading and get access to the full archive.

Continue reading