Logic App Framework for Core Connectors (2023)

We are building a framework that allows partners and solution providers to build connectors and integrate them into theMicrosoft Sustainability ManagerSolution. This article provides details on the first set of connectors we built withAzure-Logik-Appsin cooperation with the associated team. This article provides instructions on how to create a Logic Apps connector for Microsoft Sustainability Manager and share the data required to calculate sustainability and carbon footprints with customers.

key terms

You need to know the following definitions used in this article:

Offerer– A provider is a partner who offers one or more services that enable a comprehensive, preconfigured connection to a data solution. Businesses select vendors to apply to their business by contacting them. Once they connect, an instance of that provider is created and can contain more than one provider solution. Customers can connect to any vendor's solution to bring their data into Microsoft Sustainability Manager.

Connector– A connector is software integrated into the product or provided as a standalone module that allows a user to ingest (or import) data into Microsoft Sustainability Manager. Each provider can have one or more connectors for each of their solutions that a user connects to.

Logic-Apps: Azure Logic Apps is a cloud platform where you can build and run automated workflows with little to no code. Using the visual designer and choosing from predefined operations, you can quickly create a workflow that integrates and manages your applications, data, services, and systems.

The following table briefly defines the key concepts and terminology in Azure Logic Apps.

Vendor Solution Requirements

Microsoft Cloud for Sustainability and its partners will build connectors to the Microsoft Sustainability Manager solution to obtain comprehensive datasets for one or more sustainability and carbon emissions categories. Connectors must meet the following requirements:

• Be available in all regions and serve different types of customer industries and company sizes.

• Allow segregation of users for credentials (target system administrator) and login settings (Microsoft Sustainability Manager user).

• If needed, provide developer tools, a sandbox, and supporting documentation to create POC.

In addition, each connector must provide the following items:

• Activity data, baseline data or pre-calculated emissions essential for defining carbon laws and sustainability

• Ability to retrieve data from partner solutions by authenticating via user credentials

  (Video) Build Custom Connectors for Azure Logic Apps

• Ability to automate data mapping of entities and attributes

High-level design of logic app connectors

The following general layout of Logic Apps connectors provides a brief description of the ingestion flow from the user's perspective. These steps are not what you would follow to create a connector, but an overview of how to use the connector.

1. Connector selection:

   1. The user will be prompted to specify the type of data they wish to include along with the scope and category.
   2. In the next step, a list shows Power Query and custom connectors that support the data type, range and category selected by the user.

2. Selection of certificates:

   1. To create new credentials for a Logic Apps connector, users must provide the credentials.
   2. Microsoft Sustainability Manager validates the credentials using the partner-specified protocol and sends the connection request to the backend.

3. Connection and creation of triggers:

   1. To fulfill the connection requirement, Microsoft provisions a Logic Apps instance using the partner-specified ARM template. The ARM model should follow the connector paradigm described later in this article.
   2. After the Logic Apps instance is deployed, it is activated once and can be scheduled by the user to run daily. The connection setup is now complete.
   3. The user can optionally activate their own instance of Logic Apps in the user interface.

4. Record a dataverse:

   1. Each execution of the Logic Apps workflow (whether manually triggered or scheduled) sends the split data to the Sustainability microservice via an HTTPS call. The data would then be validated and integrated into the customer environment.

5. Connection Exclusion:

   1. The connection removal is handled by the Logic Apps framework. When a user deletes a Logic Apps connection, we terminate the Logic Apps instance for the specified connection and then delete the connection.
   2. Deleting the connection principal would also cascade connection update IDs.

Create a Logic Apps connector

To create a core connector in Microsoft Sustainability Manager, complete the following steps to create the connector definition, provider definition, ARM template for Logic Apps, and credential configuration. Add these files to a private GitHub repository and add our team as contributors:mcfsconnectors@microsoft.com.

Step 1: Data Verification

The first step in creating a connector is to review the data type available in the partner's solution and its application in Microsoft Sustainability Manager. These types of data can include activity data, baseline data, or pre-calculated emissions, all of which are critical to defining carbon laws and sustainability.

Step 2: Define connector and provider or partner

The next step in creating the connector involves gathering additional information about the solution that will be integrated with Microsoft Sustainability Manager.

• Provider definition:

  Information	Description
  Name	your company name.
  Description	A brief description of your business.
  Logo	Company logo in SVG format.
  URL	Company URL.

• Connector definition:

  Information	Description
  Name	Connector friendly name.
  Description	A short description of the connector in two sentences.
  Logo	Plug icon in SVG format.
  type of data	The type of data this connector ingests for the user. Example: activity data, scope 1, category 2. For more information on data types, seeRecord.

This image shows an example of what a connector might look like. This example shows the sample logo.Connection for data distributionis the display name,Johnnie McConnelis the name of the provider andOur internal datais the description.

Step 3: Define the data source

Before you can create a logic app, you must define the data source and how it can be accessed using user credentials. Here are some reference examples:

• RESTO-HTTP: You can expose an HTTP REST API to expose data to your logic app. You can use HTTP authentication mechanisms to access this data from our logic app.

• Azure Data Lake storage: <https://<adlsname>.blob.core.windows.net/<folder name>/<file name>>, in this case your logic app needs relevant access to this URL with the appropriate SAS token.

• tabla mysql: <jdbc:mysql:replication//myUser:myPassword@[address=(host=myHost1)(port=1111)(key1=value1)]> Data can be stored in a database. In this case, your logic app might need the database endpoint along with a valid user ID and password. You can also request the name of the table that stores the data to ingest. For more information on connecting to a SQL database, seeConnect to a workflow SQL database in Azure Logic Apps.

• AWS-S3: <s3.region.amazonaws.com> Data can be stored in an S3 bucket or similar file storage. The Logic Apps instance would require appropriate access control to access this storage bucket.

Step 4: Get Credentials

After you have defined the data source and access details, you must provide the credentials that must be collected from the user to authenticate to the data source.

For core connectors, Microsoft Sustainability Manager does not currently support SSO. Credentials are collected and stored in the customer's Dataverse environment.

When users create a Microsoft Sustainability Manager logic app connection, they are prompted with a page similar to the following image to enter relevant connection details between the Microsoft Sustainability Manager hosted logic app and the partner.

Microsoft creates the actual JSON file for the credentials page. You just have to answer the following questions so we can generate the file.

Step 5: Build the logic app

The next step is to create a logic app. Microsoft provides a sample ARM template that you can upload to your Azure subscription and start developing a logic app that does the following:

• Use the user credentials to authenticate to your system and fetch data offloaded from the data source.

• Apply transformations and map the data to the sustainability data model or the water data model.

  (Video) How to call WebAPI using custom connectors in Logic Apps

• Add the transformed data to a JSON array.

To automatically deploy a logic app template to Azure, you can select the followingDon't deploy Azurewhich connects you to the Azure portal and requests the subscription ID and resource group name where this logic app is deployed.

This sample logic app is a starting point for the main connector. Input to this connector is sent over the HTTPS trigger from the Microsoft Sustainability Manager client instance. This HTTP trigger contains details provided during connector configuration (including custom endpoints and credentials).

After you deploy your logic app, you can view and edit it in the app designer:

Brief description of the actions:

1. activate: Microsoft Sustainability Manager triggers the execution of the partner logic app via an HTTP trigger passing the connection details provided by the customerStep 4: Get Credentials. Partner logic apps use a subset of the following activation parameters:

   

   • Client-ID und Client-Secret: Credentials that the user enters on the Microsoft Sustainability Manager Credentials page. These parameters can vary depending on how you authenticate your data source. For example, instead of the client ID and secret, you can use the username, password, or API key. Replace them with the credentials you identified in the previous step.

   • Time of last update: This parameter is a timestamp in the UTC time zone and is used for incremental updates. Your logic app should only extract and process data theredataCreateTime>=last update time.

2. InitializeContinuationToken: This action initializes thecontinuation tokenvariable to zero. This variable is used to exit the until loop that fetches data page by page.

3. Attempt: All logic to extract and transform data must be inside the try block. If a step within the test block fails, the execution flow jumps to the catch block, where the Microsoft team adds logic to handle failure scenarios. The following image shows the actions inside the try block.

   

The Get Token action takes the user's credentials to get the access token and uses it to call the partner endpoint and connect to the data source to get the user's data. Depending on the size of the data, the data in your data source should be accessed page-by-page to avoid timeouts. In the next step, your data is mapped to the Microsoft Sustainability Manager data model.

This model uses thepurchased electricityEntity as an example to demonstrate the mapping logic, but you need to map the data to the entity supported by your connector. Once the data is mapped, store the data array in a variable and send the data to Microsoft Sustainability Manager in a POST request.PostDataToMSMis an example of how the HTTP request should be constructed.

You can send data to Microsoft Sustainability Manager in multiple batches by making multiple POST requests, or all at once, depending on the size of the data. EITHERfull revenueThe action shows what the last request to Microsoft Sustainability Manager should look like. Microsoft calls the same endpoint and sets thefull revenuevariable to true. In response to this request, Microsoft Sustainability Manager begins processing the data for inclusion in the customer environment.

The following figure shows an example of how data records should be structured:

Step 6: Upload the connector files

You must add all files created in the previous steps to a private GitHub repository and add your Microsoft team as a contributor. The repository root folder must have a version number (e.g. v1.0) and the following files must be in the folder:

• logic_apps_armtemplate.json
• Credentials_info.csv
• connector_definition.csv
• Provider_Definition.csv
• Company logo.svg
• connector_logo.svg

Your Microsoft team starts the logic app validation process and adds more actions to complete the capture process. After the validation step, Microsoft creates a Microsoft Sustainability Manager environment in which the connector is operational for the partner to test the functionality. Microsoft will also conduct its own functional testing before releasing the feature to users in preview or generally available.

Vendor or partner support requests

The partner owns the maintenance and operation of the data source. We require a contact person with a Service Level Agreement (SLA) that the Microsoft team can contact for outages and errors related to fixes. The following scenarios are examples that require the assistance of a partner:

• Change the data model: One of the most important steps in building the integration with Microsoft Sustainability Manager is mapping the data model of the partner solution to the sustainability data model. If the partner's solution data model changes, the partner must work with the Microsoft Sustainability Manager team to remap the data model or provide an updated ARM model to ensure the required data attributes can be retrieved from their solution.

• Permission Management: The primary mechanism for accessing the partner solution in Microsoft Sustainability Manager is through the partner solution credentials. If users are unable to authenticate their credentials to Microsoft Sustainability Manager, the partner will provide assistance in authenticating and updating the user's credentials.

• not working solutionNote: The partner must also support Microsoft Sustainability Manager and its users in situations where their solution is not operational or functional. Microsoft Sustainability Manager will work with the partner to determine root cause and resolution timelines so they can communicate with their end users.

• Solution Updates: The partner must provide advance notice of any major update along with the potential impact on user access to Microsoft Sustainability Manager. Prior to integrating a solution with Microsoft Sustainability Manager, the partner will provide tentative solution update cadence schedules, along with support documentation on how to extend support for Microsoft Sustainability Manager and its users when they do not access the solution provider and record data.

• test environment: The partner must provide a test environment for Microsoft Sustainability Manager to test the features before deploying them to their customers.

upgrade scenarios

For any logic app or connector configuration updates, the partner must create a new folder with the new version number (e.g. v2.0) in the root of the repository and add all files in that folder. The partner should also add a readme.md file to the folder with a brief explanation of what has been updated. Once Microsoft receives the updated files, we will follow the same process to release the first version of the connector. After the updated logic app is shared with the end user, existing connections are undeployed and the latest version is deployed. Our system handles this process by comparing the version numbers on the ARM model.

Compliance Requirements

Features and services in Microsoft products must undergo privacy, security, and compliance reviews before going into production. The Microsoft team owns and manages this process, but requires coordination with the partner team to gather the information needed to integrate your solution with Microsoft Sustainability Manager.

see also

import dataAzure Logic Apps documentationMicrosoft Sustainability ManagerMicrosoft cloud data model for sustainability