Skip to content

Commit 1c90ef2

Browse files
Charles-GagnonCharles-Gagnon
authored
Merge pull request #403 from Azure/chgagnon/updateFromTriggerBindings
Update release/LogicApps with latest from triggerbindings
2 parents 291f524 + babe505 commit 1c90ef2

File tree

24 files changed

+641
-121
lines changed

24 files changed

+641
-121
lines changed

‎Directory.Build.props‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,6 @@
1010
<DelaySign>True</DelaySign>
1111
<SignAssembly>True</SignAssembly>
1212
<AssemblyOriginatorKeyFile>$(MSBuildThisFileDirectory)/SQL2003.snk</AssemblyOriginatorKeyFile>
13+
<IsPackable>false</IsPackable>
1314
</PropertyGroup>
1415
</Project>

‎README.md‎

Lines changed: 43 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -55,9 +55,14 @@ Azure SQL bindings for Azure Functions are supported for:
5555
- [ICollector&lt;T&gt;/IAsyncCollector&lt;T&gt;](#icollectortiasynccollectort)
5656
- [Array](#array)
5757
- [Single Row](#single-row)
58-
- [Primary Keys and Identity Columns](#primary-keys-and-identity-columns)
58+
- [Primary Key Special Cases](#primary-key-special-cases)
59+
- [Identity Columns](#identity-columns)
60+
- [Columns with Default Values](#columns-with-default-values)
5961
- [Trigger Binding](#trigger-binding)
6062
- [Change Tracking](#change-tracking)
63+
- [Internal State Tables](#internal-state-tables)
64+
- [az_func.GlobalState](#az_funcglobalstate)
65+
- [az_func.Leases_*](#az_funcleases_)
6166
- [Trigger Samples](#trigger-samples)
6267
- [Known Issues](#known-issues)
6368
- [Telemetry](#telemetry)
@@ -826,20 +831,26 @@ public static IActionResult Run(
826831
}
827832
```
828833
829-
#### Primary Keys and Identity Columns
834+
#### Primary Key Special Cases
830835
831836
Normally Output Bindings require two things :
832837
833838
1. The table being upserted to contains a Primary Key constraint (composed of one or more columns)
834839
2. Each of those columns must be present in the POCO object used in the attribute
835840
836-
If either of these are false then an error will be thrown.
841+
Normally if either of these are false then an error will be thrown. Below are the situations in which this is not the case :
837842
838-
This changes if one of the primary key columns is an identity column though. In that case there are two options based on how the function defines the output object:
843+
##### Identity Columns
844+
In the case where one of the primary key columns is an identity column, there are two options based on how the function defines the output object:
839845
840846
1. If the identity column isn't included in the output object then a straight insert is always performed with the other column values. See [AddProductWithIdentityColumn](./samples/samples-csharp/OutputBindingSamples/AddProductWithIdentityColumn.cs) for an example.
841847
2. If the identity column is included (even if it's an optional nullable value) then a merge is performed similar to what happens when no identity column is present. This merge will either insert a new row or update an existing row based on the existence of a row that matches the primary keys (including the identity column). See [AddProductWithIdentityColumnIncluded](./samples/samples-csharp/OutputBindingSamples/AddProductWithIdentityColumnIncluded.cs) for an example.
842848
849+
##### Columns with Default Values
850+
In the case where one of the primary key columns has a default value, there are also two options based on how the function defines the output object:
851+
1. If the column with a default value is not included in the output object, then a straight insert is always performed with the other values. See [AddProductWithDefaultPK](./samples/samples-csharp/OutputBindingSamples/AddProductWithDefaultPK.cs) for an example.
852+
2. If the column with a default value is included then a merge is performed similar to what happens when no default column is present. If there is a nullable column with a default value, then the provided column value in the output object will be upserted even if it is null.
853+
843854
### Trigger Binding
844855
845856
> **NOTE:** Trigger binding support is only available for C# functions at present.
@@ -869,6 +880,34 @@ The trigger binding utilizes SQL [change tracking](https://docs.microsoft.com/sq
869880
870881
> **NOTE:** The leases table contains all columns corresponding to the primary key from the user table and three additional columns named `_az_func_ChangeVersion`, `_az_func_AttemptCount` and `_az_func_LeaseExpirationTime`. If any of the primary key columns happen to have the same name, that will result in an error message listing any conflicts. In this case, the listed primary key columns must be renamed for the trigger to work.
871882
883+
#### Internal State Tables
884+
885+
The trigger functionality creates several tables to use for tracking the current state of the trigger. This allows state to be persisted across sessions and for multiple instances of a trigger binding to execute in parallel (for scaling purposes).
886+
887+
In addition, a schema named `az_func` will be created that the tables will belong to.
888+
889+
The login the trigger is configured to use must be given permissions to create these tables and schema. If not, then an error will be thrown and the trigger will fail to run.
890+
891+
If the tables are deleted or modified, then unexpected behavior may occur. To reset the state of the triggers, first stop all currently running functions with trigger bindings and then either truncate or delete the tables. The next time a function with a trigger binding is started, it will recreate the tables as necessary.
892+
893+
##### az_func.GlobalState
894+
895+
This table stores information about each function being executed, what table that function is watching and what the [last sync state](https://learn.microsoft.com/sql/relational-databases/track-changes/work-with-change-tracking-sql-server) that has been processed.
896+
897+
##### az_func.Leases_*
898+
899+
A `Leases_*` table is created for every unique instance of a function and table. The full name will be in the format `Leases_<FunctionId>_<TableId>` where `<FunctionId>` is generated from the function ID and `<TableId>` is the object ID of the table being tracked. Such as `Leases_7d12c06c6ddff24c_1845581613`.
900+
901+
This table is used to ensure that all changes are processed and that no change is processed more than once. This table consists of two groups of columns:
902+
903+
* A column for each column in the primary key of the target table - used to identify the row that it maps to in the target table
904+
* A couple columns for tracking the state of each row. These are:
905+
* `_az_func_ChangeVersion` for the change version of the row currently being processed
906+
* `_az_func_AttemptCount` for tracking the number of times that a change has attempted to be processed to avoid getting stuck trying to process a change it's unable to handle
907+
* `_az_func_LeaseExpirationTime` for tracking when the lease on this row for a particular instance is set to expire. This ensures that if an instance exits unexpectedly another instance will be able to pick up and process any changes it had leases for after the expiration time has passed.
908+
909+
A row is created for every row in the target table that is modified. These are then cleaned up after the changes are processed for a set of changes corresponding to a change tracking sync version.
910+
872911
#### Trigger Samples
873912
The trigger binding takes two [arguments](https://github.com/Azure/azure-functions-sql-extension/blob/main/src/TriggerBinding/SqlTriggerAttribute.cs)
874913

‎builds/azure-pipelines/template-steps-build-test.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ steps:
3232
displayName: 'Set npm installation path for Windows'
3333
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
3434

35+
- bash: echo "##vso[task.setvariable variable=azureFunctionsExtensionBundlePath]$(func GetExtensionBundlePath)"
36+
displayName: 'Set Azure Functions extension bundle path'
37+
workingDirectory: $(Build.SourcesDirectory)/samples/samples-js
38+
3539
- task: DockerInstaller@0
3640
displayName: Docker Installer
3741
inputs:
@@ -67,6 +71,14 @@ steps:
6771
projects: '${{ parameters.solution }}'
6872
arguments: '--configuration ${{ parameters.configuration }} -p:GeneratePackageOnBuild=false -p:Version=${{ parameters.binariesVersion }}'
6973

74+
- task: CopyFiles@2
75+
displayName: 'Copy Sql extension dll to Azure Functions extension bundle'
76+
inputs:
77+
sourceFolder: $(Build.SourcesDirectory)/src/bin/${{ parameters.configuration }}/netstandard2.0
78+
contents: Microsoft.Azure.WebJobs.Extensions.Sql.dll
79+
targetFolder: $(azureFunctionsExtensionBundlePath)/bin
80+
overWrite: true
81+
7082
- script: |
7183
npm install
7284
npm run lint

‎java-library/pom.xml‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3+
<modelVersion>4.0.0</modelVersion>
4+
5+
<groupId>com.microsoft.azure.functions</groupId>
6+
<artifactId>azure-functions-java-library-sql</artifactId>
7+
<version>0.0.0</version>
8+
<packaging>jar</packaging>
9+
10+
<parent>
11+
<groupId>com.microsoft.maven</groupId>
12+
<artifactId>java-8-parent</artifactId>
13+
<version>8.0.1</version>
14+
</parent>
15+
16+
<name>Microsoft Azure Functions Java SQL Types</name>
17+
<description>This package contains all Java annotations to interact with Microsoft Azure Functions runtime for SQL Bindings.</description>
18+
<url>https://aka.ms/sqlbindings</url>
19+
<organization>
20+
<name>Microsoft Azure</name>
21+
<url>https://azure.microsoft.com</url>
22+
</organization>
23+
24+
<properties>
25+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
26+
</properties>
27+
28+
<licenses>
29+
<license>
30+
<name>MIT License</name>
31+
<url>https://opensource.org/licenses/MIT</url>
32+
<distribution>repo</distribution>
33+
</license>
34+
</licenses>
35+
36+
<scm>
37+
<connection>scm:git:https://github.com/Azure/azure-functions-sql-extension</connection>
38+
<developerConnection>scm:git:git@github.com:Azure/azure-functions-sql-extension</developerConnection>
39+
<url>https://github.com/Azure/azure-functions-sql-extension</url>
40+
<tag>HEAD</tag>
41+
</scm>
42+
43+
<developers>
44+
<developer>
45+
<id>LucyZhang</id>
46+
<name>Lucy Zhang</name>
47+
<email>luczhan@microsoft.com</email>
48+
</developer>
49+
</developers>
50+
51+
<distributionManagement>
52+
<snapshotRepository>
53+
<id>ossrh</id>
54+
<name>Sonatype Snapshots</name>
55+
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
56+
<uniqueVersion>true</uniqueVersion>
57+
<layout>default</layout>
58+
</snapshotRepository>
59+
</distributionManagement>
60+
61+
<repositories>
62+
<repository>
63+
<id>maven.snapshots</id>
64+
<name>Maven Central Snapshot Repository</name>
65+
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
66+
<releases>
67+
<enabled>false</enabled>
68+
</releases>
69+
<snapshots>
70+
<enabled>true</enabled>
71+
</snapshots>
72+
</repository>
73+
</repositories>
74+
75+
<dependencies>
76+
<dependency>
77+
<groupId>com.microsoft.azure.functions</groupId>
78+
<artifactId>azure-functions-java-library</artifactId>
79+
<version>1.4.2</version>
80+
</dependency>
81+
</dependencies>
82+
83+
<build>
84+
<plugins>
85+
<plugin>
86+
<artifactId>maven-compiler-plugin</artifactId>
87+
<version>${maven-compiler.version}</version>
88+
</plugin>
89+
<plugin>
90+
<groupId>org.apache.maven.plugins</groupId>
91+
<artifactId>maven-source-plugin</artifactId>
92+
<version>${maven-source.version}</version>
93+
<executions>
94+
<execution>
95+
<id>attach-sources</id>
96+
<goals>
97+
<goal>jar</goal>
98+
</goals>
99+
</execution>
100+
</executions>
101+
</plugin>
102+
<plugin>
103+
<groupId>org.apache.maven.plugins</groupId>
104+
<artifactId>maven-javadoc-plugin</artifactId>
105+
<version>${maven-javadoc.version}</version>
106+
<configuration>
107+
<doclint>none</doclint>
108+
</configuration>
109+
<executions>
110+
<execution>
111+
<id>attach-javadocs</id>
112+
<goals>
113+
<goal>jar</goal>
114+
</goals>
115+
</execution>
116+
</executions>
117+
</plugin>
118+
</plugins>
119+
</build>
120+
</project>

‎java-library/src/main/java/com/microsoft/azure/functions/sql/annotation/SQLInput.java‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
/**
2+
* Copyright (c) Microsoft Corporation. All rights reserved.
3+
* Licensed under the MIT License. See License.txt in the project root for
4+
* license information.
5+
*/
6+
7+
package com.microsoft.azure.functions.sql.annotation;
8+
9+
import java.lang.annotation.Retention;
10+
import java.lang.annotation.Target;
11+
import java.lang.annotation.RetentionPolicy;
12+
import java.lang.annotation.ElementType;
13+
14+
import com.microsoft.azure.functions.annotation.CustomBinding;
15+
16+
@Retention(RetentionPolicy.RUNTIME)
17+
@Target(ElementType.PARAMETER)
18+
@CustomBinding(direction = "in", name = "sql", type = "sql")
19+
public @interface SQLInput {
20+
/**
21+
* Query string or name of stored procedure to be run.
22+
*/
23+
String commandText() default "";
24+
25+
/**
26+
* Text or Stored Procedure.
27+
*/
28+
String commandType() default "";
29+
30+
/**
31+
* Parameters to the query or stored procedure. This string must follow the format
32+
* "@param1=param1,@param2=param2" where @param1 is the name of the parameter and
33+
* param1 is the parameter value.
34+
*/
35+
String parameters() default "";
36+
37+
/**
38+
* Setting name for SQL connection string.
39+
*/
40+
String connectionStringSetting() default "";
41+
}

‎java-library/src/main/java/com/microsoft/azure/functions/sql/annotation/SQLOutput.java‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/**
2+
* Copyright (c) Microsoft Corporation. All rights reserved.
3+
* Licensed under the MIT License. See License.txt in the project root for
4+
* license information.
5+
*/
6+
7+
package com.microsoft.azure.functions.sql.annotation;
8+
9+
import java.lang.annotation.Retention;
10+
import java.lang.annotation.Target;
11+
import java.lang.annotation.RetentionPolicy;
12+
import java.lang.annotation.ElementType;
13+
14+
import com.microsoft.azure.functions.annotation.CustomBinding;
15+
16+
17+
@Retention(RetentionPolicy.RUNTIME)
18+
@Target({ ElementType.PARAMETER, ElementType.METHOD })
19+
@CustomBinding(direction = "out", name = "sql", type = "sql")
20+
public @interface SQLOutput {
21+
/**
22+
* Name of the table to upsert data to.
23+
*/
24+
String commandText() default "";
25+
26+
/**
27+
* Setting name for SQL connection string.
28+
*/
29+
String connectionStringSetting() default "";
30+
}

‎performance/SqlTriggerBindingPerformance.cs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ await this.WaitForProductChanges(
3232
() => { this.InsertProducts(1, count); return Task.CompletedTask; },
3333
id => $"Product {id}",
3434
id => id * 100,
35-
GetBatchProcessingTimeout(1, count));
35+
this.GetBatchProcessingTimeout(1, count));
3636
}
3737

3838
[IterationCleanup]

‎samples/samples-csharp/Common/Product.cs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,11 @@ public class ProductName
2525
{
2626
public string Name { get; set; }
2727
}
28+
29+
public class ProductWithDefaultPK
30+
{
31+
public string Name { get; set; }
32+
33+
public int Cost { get; set; }
34+
}
2835
}

‎samples/samples-csharp/Database/Tables/ProductsWithDefaultPK.sql‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
CREATE TABLE [ProductsWithDefaultPK] (
2+
[ProductGuid] [uniqueidentifier] PRIMARY KEY NOT NULL DEFAULT(newsequentialid()),
3+
[Name] [nvarchar](100) NULL,
4+
[Cost] [int] NULL
5+
)

‎samples/samples-csharp/OutputBindingSamples/AddProductWithDefaultPK.cs‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
// Copyright (c) Microsoft Corporation. All rights reserved.
2+
// Licensed under the MIT License. See License.txt in the project root for license information.
3+
4+
using Microsoft.AspNetCore.Mvc;
5+
using Microsoft.Azure.WebJobs.Extensions.Http;
6+
using Microsoft.Azure.WebJobs.Extensions.Sql.Samples.Common;
7+
8+
namespace Microsoft.Azure.WebJobs.Extensions.Sql.Samples.OutputBindingSamples
9+
{
10+
11+
public static class AddProductWithDefaultPK
12+
{
13+
/// <summary>
14+
/// This shows an example of a SQL Output binding where the target table has a default primary key
15+
/// of type uniqueidentifier and the column is not included in the output object. A new row will
16+
/// be inserted and the uniqueidentifier will be generated by the engine.
17+
/// </summary>
18+
/// <param name="req">The original request that triggered the function</param>
19+
/// <param name="product">The created ProductWithDefaultPK object</param>
20+
/// <returns>The CreatedResult containing the new object that was inserted</returns>
21+
[FunctionName(nameof(AddProductWithDefaultPK))]
22+
public static IActionResult Run(
23+
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "addproductwithdefaultpk")]
24+
[FromBody] ProductWithDefaultPK product,
25+
[Sql("dbo.ProductsWithDefaultPK", ConnectionStringSetting = "SqlConnectionString")] out ProductWithDefaultPK output)
26+
{
27+
output = product;
28+
return new CreatedResult($"/api/addproductwithdefaultpk", output);
29+
}
30+
}
31+
}

0 commit comments

Comments
 (0)