How to onboard thousand of devices on AWS Greengrass and live happily (2023)

How to onboard thousand of devices on AWS Greengrass and live happily (1)

"If I had six hours to chop down a tree, I'd spend the first four hours sharpening the axe." Abraham Lincoln.

I must confess: I'm lazy, so lazy that I spend my time working on automating everything. My fellow colleagues know my attitude, and I hope to avoid ending in this situation:

How to onboard thousand of devices on AWS Greengrass and live happily (2)

When it comes to massive deployments, automation is the only strategy that guarantees survival in the IT world.

A customer asked us to help him use its extra edge computational capacity spread over the country. Every branch had a virtualization environment: running on-demand workloads with the least possible maintenance and configuration effort would be the icing on the cake.

Since everything is on the edge and networking communication could be tricky, we thought about AWS Greengrass. Mattia already used Greengrass to deploy long-running and on-demand functions at the edge; we were confident that the technology could fit our case. Our last problem to solve was avoiding to onboard every virtual machine manually.

Lucky for us, Greengrass Fleet provisioning is the solution we were searching for. It allows massive deployments of edge devices, maintaining security over the enrollment process.

To automatically provision a fleet of devices, we need first to authenticate our request during the onboarding phase; otherwise, everyone could add themselves and do evil things inside our deployment. Once the provisioning phase is completed, a unique certificate will be assigned to that device and will be used as the authentication method to interact with AWS Services.

Two authentication methods (claims and trusted users) are available; their usage depends on the deployment scenario.

(Video) IoT at the Edge: Greengrass and More

Provisioning using a trusted user applies to scenarios when user onboarding needs user interaction (like wifi-connected appliances with companion mobile apps).

It impersonates a user who requests a certificate on behalf of the IoT device, then uploads the final certificate to the device.

Provisioning using claims uses a "base" certificate (stored on the device) to authenticate and request the final certificate directly from the IoT device. This scenario requires no user interaction, so massive deployments are a perfect fit.

Bingo! We have our solution: we will use the AWS IoT Greengrass Core with the AWS IoT fleet provisioning plugin.

First, let's dig into the claims provisioning workflow to understand it better.

  1. Create a provisioning certificate and allow it to obtain final certificates by attaching a policy
  2. Create a provisioning template that automatically fills device details (like name, group, or serial number) in AWS IoT Core
  3. Install Greengrass software on a template edge device,
  4. Configure the fleet provisioning plugin and customize the deployment.
  5. Start deploying devices!

Since we are lazy, we will use scripting and CloudFormation as much as possible. As a bonus point, we'll create a golden virtual machine image with all the installed components we can use as a template to deploy at the edge.

First, a script will create and save provisioning certificates using AWS CLI. We will save them using AWS SecretsManager, as you will see later, we use SecretsManager to automate their deployment on the golden virtual machine image.

#!/bin/bashENV=${1}SECRETSMANAGER_PREFIX="MyAwesomeProjectSecret"THINGGROUP_NAME="MyThingGroup"# Create the certificateCERTIFICATE_ARN=$(aws iot create-keys-and-certificate \ --certificate-pem-outfile "claim-certs/claim.pem.crt" \ --public-key-outfile "claim-certs/claim.public.pem.key" \ --private-key-outfile "claim-certs/claim.private.pem.key" \ --set-as-active | jq -r .certificateArn)SECRET_STRING=$(echo "$(cat claim-certs/claim.pem.crt)|$(cat claim-certs/claim.private.pem.key)" | tr '\n' ';' )aws secretsmanager create-secret --name $SECRETSMANAGER_PREFIX/$ENV/claims-certificate --secret-string $SECRET_STRING# Attach the policy to the certificateaws iot attach-policy --policy-name GreengrassProvisioningClaimPolicy --target $CERTIFICATE_ARNaws iot create-thing-group --thing-group-name $THINGGROUP_NAME

Run this script before deploying this CloudFormation template that will create the thing group, policies to associate it with claim certificates, and the required service roles.

---AWSTemplateFormatVersion: '2010-09-09'Description: GreenGrass with Fleet Manager provisioning templateMetadata: 'AWS::CloudFormation::Interface':ParameterGroups: - Label: {default: 'Required parameters'} Parameters: - Env - ThingGroupName - Label: {default: 'Optional parameters'} Parameters: - ProjectNameParameters: Env:Type: StringDescription: "Insert the environment" ProjectName:Type: StringDescription: "Insert the name of the project" ThingGroupName:Type: StringDescription: "Insert the Thing Group name"Resources: GreengrassTokenExchangeRole:Type: AWS::IAM::RoleProperties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: 'Allow' Principal: Service: - '' Action: - 'sts:AssumeRole' - Effect: 'Allow' Policies: - PolicyName: 'LogPersmission' PolicyDocument: Version: '2012-10-17' Statement: - Effect: 'Allow' Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" - "logs:DescribeLogStreams" - "s3:GetBucketLocation" - "iam:PassRole" Resource: '*' RoleName: !Sub '${ProjectName}-${Env}-V2TokenExchangeRole' GreengrassTokenExchangeRoleAlias:Type: AWS::IoT::RoleAliasProperties: RoleAlias: !Sub '${ProjectName}-${Env}-CoreTokenExchangeRoleAlias' RoleArn: !GetAtt GreengrassTokenExchangeRole.Arn Tags: - Key: Name Value: !Sub "${ProjectName}-${Env}-CoreTokenExchangeRoleAlias" GreengrassV2IoTThingPolicy:Type: AWS::IoT::PolicyProperties: PolicyDocument: Version: '2012-10-17' Statement: - Effect: 'Allow' Action: - "iot:Publish" - "iot:Subscribe" - "iot:Receive" - "iot:Connect" - "greengrass:*" Resource: "*" - Effect: 'Allow' Action: - "iot:AssumeRoleWithCertificate" Resource: !GetAtt GreengrassTokenExchangeRoleAlias.RoleAliasArn PolicyName: V2IoTThingPolicy GreengrassFleetProvisioningRole:Type: AWS::IAM::RoleProperties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: 'Allow' Principal: Service: - '' Action: - 'sts:AssumeRole' Policies: - PolicyName: 'ECRAccess' PolicyDocument: Version: '2012-10-17' Statement: - Effect: 'Allow' Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" - "logs:DescribeLogStreams" - "s3:GetBucketLocation" Resource: '*' RoleName: !Sub '${ProjectName}-${Env}-FleetProvisioningRole' GreengrassFleetProvisioningTemplate:Type: AWS::IoT::ProvisioningTemplateProperties: Description: "template to provision iot fleet resources" Enabled: true ProvisioningRoleArn: !GetAtt GreengrassFleetProvisioningRole.Arn TemplateBody: | { "Parameters": { "ThingName": { "Type": "String" }, "ThingGroupName": { "Type": "String" }, "AWS::IoT::Certificate::Id": { "Type": "String" } }, "Resources": { "MyThing": { "OverrideSettings": { "AttributePayload": "REPLACE", "ThingGroups": "REPLACE", "ThingTypeName": "REPLACE" }, "Properties": { "AttributePayload": {}, "ThingGroups": [ { "Ref": "ThingGroupName" } ], "ThingName": { "Ref": "ThingName" } }, "Type": "AWS::IoT::Thing" }, "MyPolicy": { "Properties": { "PolicyName": "V2IoTThingPolicy" }, "Type": "AWS::IoT::Policy" }, "MyCertificate": { "Properties": { "CertificateId": { "Ref": "AWS::IoT::Certificate::Id" }, "Status": "Active" }, "Type": "AWS::IoT::Certificate" } } } TemplateName: !Sub "${ProjectName}-${Env}-FleetTemplate" GreengrassProvisioningClaimPolicy:Type: AWS::IoT::PolicyProperties: PolicyDocument: Version: '2012-10-17' Statement: - Effect: 'Allow' Action: - "iot:Connect" Resource: "*" - Effect: 'Allow' Action: - "iot:Subscribe" Resource: - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topicfilter/$aws/certificates/create/*" - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topicfilter/$aws/provisioning-templates/${GreengrassFleetProvisioningTemplate}/provision/*" - Effect: 'Allow' Action: - "iot:Publish" - "iot:Receive" Resource: - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topic/$aws/certificates/create/*" - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topic/$aws/provisioning-templates/${GreengrassFleetProvisioningTemplate}/provision/*" PolicyName: GreengrassProvisioningClaimPolicy


(Video) "Get all your (IoT) Ducks in a Row with AWS Greengrass" by Calvin Hendryx-Parker


resource will register our IoT device using a GroupName and a ThingName. You can personalize the template according to your needs. Later, you will see that "ThingName" will be our VM hostname.

If you want to get notified and run custom actions when a device registers, you can add a rule like this and the corresponding SNS topics (I will not include the complete code since it's a simple lambda function)

ThingCreationIotRule:Type: AWS::IoT::TopicRuleProperties: RuleName: !Sub '${ProjectName}_${Env}_greengrass_rule' TopicRulePayload: RuleDisabled: 'false' Sql: SELECT * FROM '$aws/events/thing/#' Actions: - Lambda: FunctionArn: !GetAtt RegisterThingLambda.Arn ErrorAction: Sns: TargetArn: !Ref SNSFailedNotificationTopic RoleArn: !GetAtt NotificationRole.Arn

Now it's time to create our golden virtual machine image using Ubuntu 22.04 as a base install; you use VirtualBox or VMware as a testing and development environment.

The following script will install the required packages and retrieve claim certificates. To use it, you should set temporary AWS IAM credentials as environment variables or, for better, integrate everything in a packer template.

#!/bin/bashENV=${1}SECRETSMANAGER_PREFIX="MyAwesomeProjectSecret"mkdir -p claim-certsSECRET_STRING=$(aws secretsmanager get-secret-value --secret-id $SECRETSMANAGER_PREFIX/$ENV/claims-certificate --query SecretString --output text)echo $SECRET_STRING | cut -d '|' -f 1 | tr ';' '\n' > claim-certs/claim.pem.crtecho $SECRET_STRING | cut -d '|' -f 2 | tr ';' '\n' > claim-certs/claim.private.pem.keyexport DEBIAN_FRONTEND=noninteractiveapt update && apt upgrade -yapt -yq install python3-pip zip open-vm-tools default-jdk libgl1-mesa-glxmkdir -p /greengrass/v2/installermkdir /greengrass/v2/claim-certsmv claim-certs/claim.private.pem.key /greengrass/v2/claim-certs/mv /claim-certs/claim.pem.crt /greengrass/v2/claim-certs/cd /greengrass/v2curl -o /greengrass/v2/AmazonRootCA1.pem -s > greengrass-nucleus-latest.zipunzip -d installer && rm greengrass-nucleus-latest.zipcurl -s > installer/aws.greengrass.FleetProvisioningByClaim.jar

All the required Greengrass components are now installed. Let's now add the ultimate lazy sysadmin touch to our solution: we definitely want to have something that, when started, configures itself without user intervention.

We will define a template for our Greengrass configuration and, taking advantage of


unit and scripting, use the virtual machine's hostname as ThingName
Create a file named config.yml in


specifying that we will use the IoT fleet provisioning plugin.

---services: aws.greengrass.Nucleus:version: "2.9.1" aws.greengrass.FleetProvisioningByClaim:configuration: rootPath: "/greengrass/v2" awsRegion: "eu-west-1" iotDataEndpoint: "" iotCredentialEndpoint: "" iotRoleAlias: "myproject-environment-CoreTokenExchangeRoleAlias" provisioningTemplate: "myproject-environment-FleetTemplate" claimCertificatePath: "/greengrass/v2/claim-certs/claim.pem.crt" claimCertificatePrivateKeyPath: "/greengrass/v2/claim-certs/claim.private.pem.key" rootCaPath: "/greengrass/v2/AmazonRootCA1.pem" templateParameters: ThingName: "REPLACEME" ThingGroupName: "MyThingGroup"

Change this file according to your environment. You can obtain

(Video) AWS re:Invent 2017: NEW LAUNCH! AWS Greengrass and Amazon FreeRTOS... Integrating th (IOT403)




values by issuing these commands:

aws iot describe-endpoint --endpoint-type iot:Data-ATSaws iot describe-endpoint --endpoint-type iot:CredentialProvider

As you can see, ThingName value is set as "REPLACEME". A script


will replace this value and a systemd service will run it at the first boot.

#!/bin/bashset -esed -i s/REPLACEME/$(hostname)/g /greengrass/v2/installer/config.yamlecho "********************** RUN SERVICE **************************"java -Droot="/greengrass/v2" \-jar /greengrass/v2/installer/lib/Greengrass.jar \--trusted-plugin /greengrass/v2/installer/aws.greengrass.FleetProvisioningByClaim.jar \--init-config /greengrass/v2/installer/config.yaml \--component-default-user ggc_user:ggc_group \--setup-system-service true

The first line

(sed -i s/REPLACEME/$(hostname)/g /greengrass/v2/installer/config.yaml) 

replaces the value with the virtual machine's current hostname.

Create now a unit file in the /lib/systemd/system directory with a meaningful name (we used "greengrass-config.service")

[Unit]Before=systemd-user-sessions.serviceWants=network-online.targetAfter=network-online.targetConditionPathExists=!/greengrass/greengrass_installed[Service]Type=oneshotExecStart=/greengrass/configure-device.shExecStartPost=/usr/bin/touch /greengrass/greengrass_installedRemainAfterExit=yes[Install]

This is a pretty standard systemd unit, with an exception:


tells systemd to run this unit only if this file does not exist.

(Video) AWS re:Invent 2018: [NEW LAUNCH!] Introducing New AWS IoT Greengrass Features (IOT365)

ExecStartPost=/usr/bin/touch /greengrass/greengrass_installed 

tells systemd to create the file that, so after the first time, the service will not run anymore.

Activate service at boot with

systemctl daemon-reloadsystemctl enable greengrass-config.service

Once you finish this configuration, just export your virtual machine image in a format such as OVA (or create a VMware template) and deploy it!

Remember: this is only a sample configuration; you may need to optimize and update your system to shrink disk size and disable unnecessary system services.

Once you deploy this solution, you can run functions on your distributed environment and take advantage of all the unused computing resources you may have at your disposal on edge.

Other AWS Services can use this solution as a base, like running managed SageMaker jobs at the edge, but let's keep this topic for another time.

Today we saw how automation could relieve us from the execution of boring tasks, even if, at first glance, everything seems to get more complicated.

Did you ever find yourself in an automation frenzy to accommodate laziness?

Let us know in the comments!

(Video) IoT All the Things | S3 E1 | All in with James Gosling: Behind the Scenes with AWS IoT Greengrass V2

See you in 14 days on #Proud2beCloud

About Proud2beCloud

Proud2beCloud is a blog bybeSharp, an Italian APN Premier Consulting Partner expert in designing, implementing, and managing complex Cloud infrastructures and advanced services on AWS. Before being writers, we are Cloud Experts working daily with AWS services since 2007. We are hungry readers, innovative builders, and gem-seekers. On Proud2beCloud, we regularly share our best AWS pro tips, configuration insights, in-depth news, tips&tricks, how-tos, and many other resources. Take part in the discussion!


Which protocol can be used in AWS IoT greengrass for routing between devices? ›

AWS IoT Greengrass uses the Application Layer Protocol Network (ALPN) TLS extension to enable these connections. For more information, see OTA updates of AWS IoT Greengrass Core software and Connect on port 443 or through a network proxy.

What are the advantages of greengrass AWS? ›

AWS IoT Greengrass enables local processing, messaging, data management, ML inference, and offers prebuilt components to accelerate application development. AWS IoT Greengrass also provides a secure way to seamlessly connect your edge devices to any AWS service as well as to third-party services.

What is the difference between IoT core and greengrass? ›

IoT Core is a cloud service but Greengrass is an edge runtime, with supporting cloud service. You can build your device software on top of the Greengrass edge runtime. For devices with sufficient resources, Greengrass is a great way to compose and manage your device software.

What are the two protocols used most often with IoT device? ›

IoT protocols can be divided into two categories: IoT network protocols and IoT data protocols. Data protocols mainly focus on information exchange, while network protocols provide methods of connecting IoT edge devices with other edge devices or the Internet.

What are two protocols used often with IoT devices? ›

Most common protocols
  1. AMQP. Short for Advanced Message Queuing Protocol, AMQP is an open standard protocol used for more message-oriented middleware. ...
  2. Bluetooth and BLE. Bluetooth is a short-range wireless technology that uses short-wavelength, ultrahigh-frequency radio waves. ...
  3. Cellular. ...
  4. CoAP. ...
  5. DDS. ...
  6. LoRa and LoRaWAN. ...
  7. LWM2M. ...
  8. MQTT.
Mar 30, 2022

What are the functions of greengrass? ›

AWS IoT Greengrass provides a standard mechanism to process data streams, manage local data-retention policies, and transmit device data to AWS cloud services such as Amazon Simple Storage Service (Amazon S3), Amazon Kinesis, AWS IoT Core, and AWS IoT Analytics.

What is the AWS IoT Greengrass SLA percentage? ›

Service Credits
Monthly Uptime PercentageService Credit Percentage
Less than 99.9% but greater than or equal to 99.0%10%
Less than 99.0% but greater than or equal to 95.0%25%
Less than 95.0%100%
May 2, 2022

What is greengrass Lambda? ›

AWS IoT Greengrass provides a containerized Lambda runtime environment for user-defined code that you author in AWS Lambda. Lambda functions that are deployed to an AWS IoT Greengrass core run in the core's local Lambda runtime.

Which is the routing protocol used in IoT? ›

Channel-Aware Routing Protocol (CARP) is a distributed routing protocol designed for underwater communication. It can be used for IoT due to its lightweight packets.

What protocol does AWS IoT use? ›

AWS IoT Core supports standard communication protocols (HTTP, MQTT, and WebSockets and LoRaWAN are supported currently). Communication is secured using TLS. Processing data sent from connected devices.

Which are the routing protocols for IoT? ›

In this project a simulation of “IoT” network that serve the “IoT” applications using different routing protocols such as ““RIP””, ““OSPF”' and “EIGRB” and an evaluation to the performance of each routing technique was to decide which is the best routing for “IoT” applications.

What protocol does Google Cloud IoT offer to connect with linked devices? ›

Cloud IoT Core supports two protocols for device connection and communication: MQTT and HTTP.


1. AWS re:Invent 2020: Connect today, transform tomorrow with AWS IoT
(AWS Events)
2. AWS re:Invent 2016: NEW LAUNCH! Introducing AWS Greengrass (IOT201)
(Amazon Web Services)
3. IoT Compute at the Edge with AWS Greengrass • Boaz Ziniman • GOTO 2018
(GOTO Conferences)
4. Building Next-Gen Consumer Products with AWS IoT Edge Services - AWS Online Tech Talks
(AWS Online Tech Talks)
5. Developing and Deploying an Internet of Things (IoT) with AWS
6. How to Build Massive IoT with Klika Tech, Wirepas and AWS
(Klika Tech)
Top Articles
Latest Posts
Article information

Author: Twana Towne Ret

Last Updated: 25/06/2023

Views: 6017

Rating: 4.3 / 5 (64 voted)

Reviews: 87% of readers found this page helpful

Author information

Name: Twana Towne Ret

Birthday: 1994-03-19

Address: Apt. 990 97439 Corwin Motorway, Port Eliseoburgh, NM 99144-2618

Phone: +5958753152963

Job: National Specialist

Hobby: Kayaking, Photography, Skydiving, Embroidery, Leather crafting, Orienteering, Cooking

Introduction: My name is Twana Towne Ret, I am a famous, talented, joyous, perfect, powerful, inquisitive, lovely person who loves writing and wants to share my knowledge and understanding with you.