Detect and Analyze Faces with Azure AI | by Think Different – Dhiraj Patra | Jun, 2024

The flexibility to detect and analyze human faces is a core AI functionality. On this train, you’ll discover two Azure AI Companies that you should use to work with faces in pictures: the Azure AI Imaginative and prescient service, and the Face service.

Essential: This lab will be accomplished with out requesting any further entry to restricted options.

Word: From June twenty first 2022, capabilities of Azure AI providers that return personally identifiable data are restricted to prospects who’ve been granted limited access. Moreover, capabilities that infer emotional state are now not out there. For extra particulars concerning the adjustments Microsoft has made, and why — see Responsible AI investments and safeguards for facial recognition.

In case you have not already carried out so, you should clone the code repository for this course:

1. Begin Visible Studio Code.
2. Open the palette (SHIFT+CTRL+P) and run a Git: Clone command to clone the https://github.com/MicrosoftLearning/mslearn-ai-vision repository to a neighborhood folder (it does not matter which folder).
3. When the repository has been cloned, open the folder in Visible Studio Code.
4. Wait whereas further recordsdata are put in to help the C# code initiatives within the repo.

• Word: In case you are prompted so as to add required property to construct and debug, choose Not Now.

If you happen to don’t have already got one in your subscription, you’ll have to provision an Azure AI Companies useful resource.

1. Open the Azure portal at https://portal.azure.com, and check in utilizing the Microsoft account related along with your Azure subscription.
2. Within the prime search bar, seek for Azure AI providers, choose Azure AI Companies, and create an Azure AI providers multi-service account useful resource with the next settings:

• Subscription: Your Azure subscription
• Useful resource group: Select or create a useful resource group (in case you are utilizing a restricted subscription, chances are you’ll not have permission to create a brand new useful resource group — use the one offered)
• Area: Select any out there area
• Title: Enter a singular identify
• Pricing tier: Normal S0

1. Choose the required checkboxes and create the useful resource.
2. Look forward to deployment to finish, after which view the deployment particulars.
3. When the useful resource has been deployed, go to it and examine its Keys and Endpoint web page. You will have the endpoint and one of many keys from this web page within the subsequent process.

On this train, you’ll full {a partially} applied shopper utility that makes use of the Azure AI Imaginative and prescient SDK to research faces in a picture.

Word: You’ll be able to select to make use of the SDK for both C# or Python. Within the steps beneath, carry out the actions applicable to your most popular language.

1. In Visible Studio Code, within the Explorer pane, browse to the 04-face folder and develop the C-Sharp or Python folder relying in your language choice.
2. Proper-click the computer-vision folder and open an built-in terminal. Then set up the Azure AI Imaginative and prescient SDK bundle by working the suitable command to your language choice:
3. C#

• dotnet add bundle Azure.AI.Imaginative and prescient.ImageAnalysis -v 0.15.1-beta.1

1. Python

• pip set up azure-ai-vision==0.15.1b1

1. View the contents of the computer-vision folder, and be aware that it comprises a file for configuration settings:

• C#: appsettings.json
• Python: .env

1. Open the configuration file and replace the configuration values it comprises to replicate the endpoint and an authentication key to your Azure AI providers useful resource. Save your adjustments.
2. Word that the computer-vision folder comprises a code file for the shopper utility:

• C#: Program.cs
• Python: detect-people.py

1. Open the code file and on the prime, below the present namespace references, discover the remark Import namespaces. Then, below this remark, add the next language-specific code to import the namespaces you will have to make use of the Azure AI Imaginative and prescient SDK:

C#

• // import namespaces utilizing Azure.AI.Imaginative and prescient.Frequent; utilizing Azure.AI.Imaginative and prescient.ImageAnalysis;

Python

• # import namespaces import azure.ai.imaginative and prescient as sdk

On this train, you’ll use the Azure AI Imaginative and prescient service to research a picture of individuals.

1. In Visible Studio Code, develop the computer-vision folder and the pictures folder it comprises.
2. Choose the folks.jpg picture to view it.

Now you’re prepared to make use of the SDK to name the Imaginative and prescient service and detect faces in a picture.

1. Within the code file to your shopper utility (Program.cs or detect-people.py), within the Major operate, be aware that the code to load the configuration settings has been offered. Then discover the remark Authenticate Azure AI Imaginative and prescient shopper. Then, below this remark, add the next language-specific code to create and authenticate a Azure AI Imaginative and prescient shopper object:

C#

• // Authenticate Azure AI Imaginative and prescient shopper var cvClient = new VisionServiceOptions( aiSvcEndpoint, new AzureKeyCredential(aiSvcKey));

Python

• # Authenticate Azure AI Imaginative and prescient shopper cv_client = sdk.VisionServiceOptions(ai_endpoint, ai_key)

1. Within the Major operate, below the code you simply added, be aware that the code specifies the trail to a picture file after which passes the picture path to a operate named AnalyzeImage. This operate just isn’t but absolutely applied.
2. Within the AnalyzeImage operate, below the remark Specify options to be retrieved (PEOPLE), add the next code:

C#

• // Specify options to be retrieved (PEOPLE) Options = ImageAnalysisFeature.Folks

Python

• # Specify options to be retrieved (PEOPLE) analysis_options = sdk.ImageAnalysisOptions() options = analysis_options.options = ( sdk.ImageAnalysisFeature.PEOPLE )

1. Within the AnalyzeImage operate, below the remark Get picture evaluation, add the next code:

C#

• // Get picture evaluation utilizing var imageSource = VisionSource.FromFile(imageFile); utilizing var analyzer = new ImageAnalyzer(serviceOptions, imageSource, analysisOptions); var consequence = analyzer.Analyze(); if (consequence.Motive == ImageAnalysisResultReason.Analyzed) { // Get folks within the picture if (consequence.Folks != null) { Console.WriteLine($" Folks:");
• // Put together picture for drawing System.Drawing.Picture picture = System.Drawing.Picture.FromFile(imageFile); Graphics graphics = Graphics.FromImage(picture); Pen pen = new Pen(Coloration.Cyan, 3); Font font = new Font("Arial", 16); SolidBrush brush = new SolidBrush(Coloration.WhiteSmoke); foreach (var particular person in consequence.Folks) { // Draw object bounding field if confidence > 50% if (particular person.Confidence > 0.5) { // Draw object bounding field var r = particular person.BoundingBox; Rectangle rect = new Rectangle(r.X, r.Y, r.Width, r.Top); graphics.DrawRectangle(pen, rect); // Return the arrogance of the particular person detected Console.WriteLine($" Bounding field {particular person.BoundingBox}, Confidence {particular person.Confidence:0.0000}"); } } // Save annotated picture String output_file = "detected_people.jpg"; picture.Save(output_file); Console.WriteLine(" Outcomes saved in " + output_file + "n"); }
• } else { var errorDetails = ImageAnalysisErrorDetails.FromResult(consequence); Console.WriteLine(" Evaluation failed."); Console.WriteLine($" Error purpose : {errorDetails.Motive}"); Console.WriteLine($" Error code : {errorDetails.ErrorCode}"); Console.WriteLine($" Error message: {errorDetails.Message}n"); }

Python

• # Get picture evaluation picture = sdk.VisionSource(image_file) image_analyzer = sdk.ImageAnalyzer(cv_client, picture, analysis_options) consequence = image_analyzer.analyze() if consequence.purpose == sdk.ImageAnalysisResultReason.ANALYZED: # Get folks within the picture if consequence.folks just isn't None: print("nPeople in picture:")
• # Put together picture for drawing picture = Picture.open(image_file) fig = plt.determine(figsize=(picture.width/100, picture.peak/100)) plt.axis('off') draw = ImageDraw.Draw(picture) coloration = 'cyan' for detected_people in consequence.folks: # Draw object bounding field if confidence > 50% if detected_people.confidence > 0.5: # Draw object bounding field r = detected_people.bounding_box bounding_box = ((r.x, r.y), (r.x + r.w, r.y + r.h)) draw.rectangle(bounding_box, define=coloration, width=3) # Return the arrogance of the particular person detected print(" {} (confidence: {:.2f}%)".format(detected_people.bounding_box, detected_people.confidence * 100)) # Save annotated picture plt.imshow(picture) plt.tight_layout(pad=0) outputfile = 'detected_people.jpg' fig.savefig(outputfile) print(' Outcomes saved in', outputfile)
• else: error_details = sdk.ImageAnalysisErrorDetails.from_result(consequence) print(" Evaluation failed.") print(" Error purpose: {}".format(error_details.purpose)) print(" Error code: {}".format(error_details.error_code)) print(" Error message: {}".format(error_details.message))

1. Save your adjustments and return to the built-in terminal for the computer-vision folder, and enter the next command to run this system:

C#

Python

1. Observe the output, which ought to point out the variety of faces detected.
2. View the detected_people.jpg file that’s generated in the identical folder as your code file to see the annotated faces. On this case, your code has used the attributes of the face to label the situation of the highest left of the field, and the bounding field coordinates to attract a rectangle round every face.

Whereas the Azure AI Imaginative and prescient service affords fundamental face detection (together with many different picture evaluation capabilities), the Face service gives extra complete performance for facial evaluation and recognition.

1. In Visible Studio Code, within the Explorer pane, browse to the 04-face folder and develop the C-Sharp or Python folder relying in your language choice.
2. Proper-click the face-api folder and open an built-in terminal. Then set up the Face SDK bundle by working the suitable command to your language choice:

C#

• dotnet add bundle Microsoft.Azure.CognitiveServices.Imaginative and prescient.Face --version 2.8.0-preview.3
• pip set up azure-cognitiveservices-vision-face==0.6.0

1. View the contents of the face-api folder, and be aware that it comprises a file for configuration settings:

• C#: appsettings.json
• Python: .env

1. Open the configuration file and replace the configuration values it comprises to replicate the endpoint and an authentication key to your Azure AI providers useful resource. Save your adjustments.
2. Word that the face-api folder comprises a code file for the shopper utility:

• C#: Program.cs
• Python: analyze-faces.py

1. Open the code file and on the prime, below the present namespace references, discover the remark Import namespaces. Then, below this remark, add the next language-specific code to import the namespaces you will have to make use of the Imaginative and prescient SDK:

C#

• // Import namespaces utilizing Microsoft.Azure.CognitiveServices.Imaginative and prescient.Face; utilizing Microsoft.Azure.CognitiveServices.Imaginative and prescient.Face.Fashions;

Python

• # Import namespaces from azure.cognitiveservices.imaginative and prescient.face import FaceClient from azure.cognitiveservices.imaginative and prescient.face.fashions import FaceAttributeType from msrest.authentication import CognitiveServicesCredentials

1. Within the Major operate, be aware that the code to load the configuration settings has been offered. Then discover the remark Authenticate Face shopper. Then, below this remark, add the next language-specific code to create and authenticate a FaceClient object:

C#

• // Authenticate Face shopper ApiKeyServiceClientCredentials credentials = new ApiKeyServiceClientCredentials(cogSvcKey); faceClient = new FaceClient(credentials) { Endpoint = cogSvcEndpoint };

Python

• # Authenticate Face shopper credentials = CognitiveServicesCredentials(cog_key) face_client = FaceClient(cog_endpoint, credentials)

1. Within the Major operate, below the code you simply added, be aware that the code shows a menu that allows you to name features in your code to discover the capabilities of the Face service. You’ll implement these features within the the rest of this train.

One of the elementary capabilities of the Face service is to detect faces in a picture, and decide their attributes, comparable to head pose, blur, the presence of spectacles, and so forth.

1. Within the code file to your utility, within the Major operate, study the code that runs if the consumer selects menu choice 1. This code calls the DetectFaces operate, passing the trail to a picture file.
2. Discover the DetectFaces operate within the code file, and below the remark Specify facial options to be retrieved, add the next code:

C#

• // Specify facial options to be retrieved IList<FaceAttributeType> options = new FaceAttributeType[] { FaceAttributeType.Occlusion, FaceAttributeType.Blur, FaceAttributeType.Glasses };

Python

• # Specify facial options to be retrieved options = [FaceAttributeType.occlusion, FaceAttributeType.blur, FaceAttributeType.glasses]

1. Within the DetectFaces operate, below the code you simply added, discover the remark Get faces and add the next code:

C#

// Get faces
utilizing (var imageData = File.OpenRead(imageFile))
{    
var detected_faces = await faceClient.Face.DetectWithStreamAsync(imageData, returnFaceAttributes: options, returnFaceId: false);

    if (detected_faces.Rely() > 0)
{
Console.WriteLine($"{detected_faces.Rely()} faces detected.");        // Put together picture for drawing
Picture picture = Picture.FromFile(imageFile);
Graphics graphics = Graphics.FromImage(picture);
Pen pen = new Pen(Coloration.LightGreen, 3);
Font font = new Font("Arial", 4);
SolidBrush brush = new SolidBrush(Coloration.White);
int faceCount=0;        // Draw and annotate every face
foreach (var face in detected_faces)
{
faceCount++;
Console.WriteLine($"nFace quantity {faceCount}");            // Get face properties
Console.WriteLine($" - Mouth Occluded: {face.FaceAttributes.Occlusion.MouthOccluded}");
Console.WriteLine($" - Eye Occluded: {face.FaceAttributes.Occlusion.EyeOccluded}");
Console.WriteLine($" - Blur: {face.FaceAttributes.Blur.BlurLevel}");
Console.WriteLine($" - Glasses: {face.FaceAttributes.Glasses}");            // Draw and annotate face
var r = face.FaceRectangle;
Rectangle rect = new Rectangle(r.Left, r.Prime, r.Width, r.Top);
graphics.DrawRectangle(pen, rect);
string annotation = $"Face quantity {faceCount}";
graphics.DrawString(annotation,font,brush,r.Left, r.Prime);
}        // Save annotated picture
String output_file = "detected_faces.jpg";
picture.Save(output_file);
Console.WriteLine(" Outcomes saved in " + output_file);   
}
}

Python

# Get faces
with open(image_file, mode="rb") as image_data:
detected_faces = face_client.face.detect_with_stream(picture=image_data,
return_face_attributes=options,                     return_face_id=False)

    if len(detected_faces) > 0:
print(len(detected_faces), 'faces detected.')        # Put together picture for drawing
fig = plt.determine(figsize=(8, 6))
plt.axis('off')
picture = Picture.open(image_file)
draw = ImageDraw.Draw(picture)
coloration = 'lightgreen'
face_count = 0        # Draw and annotate every face
for face in detected_faces:            # Get face properties
face_count += 1
print('nFace quantity {}'.format(face_count))            detected_attributes = face.face_attributes.as_dict()
if 'blur' in detected_attributes:
print(' - Blur:')
for blur_name in detected_attributes['blur']:
print('   - {}: {}'.format(blur_name, detected_attributes['blur'][blur_name]))            if 'occlusion' in detected_attributes:
print(' - Occlusion:')
for occlusion_name in detected_attributes['occlusion']:
print('   - {}: {}'.format(occlusion_name, detected_attributes['occlusion'][occlusion_name]))            if 'glasses' in detected_attributes:
print(' - Glasses:{}'.format(detected_attributes['glasses']))            # Draw and annotate face
r = face.face_rectangle
bounding_box = ((r.left, r.prime), (r.left + r.width, r.prime + r.peak))
draw = ImageDraw.Draw(picture)
draw.rectangle(bounding_box, define=coloration, width=5)
annotation = 'Face quantity {}'.format(face_count)
plt.annotate(annotation,(r.left, r.prime), backgroundcolor=coloration)        # Save annotated picture
plt.imshow(picture)
outputfile = 'detected_faces.jpg'
fig.savefig(outputfile)        print('nResults saved in', outputfile)

1. Look at the code you added to the DetectFaces operate. It analyzes a picture file and detects any faces it comprises, together with attributes for occlusion, blur, and the presence of spectacles. The main points of every face are displayed, together with a singular face identifier that’s assigned to every face; and the situation of the faces is indicated on the picture utilizing a bounding field.
2. Save your adjustments and return to the built-in terminal for the face-api folder, and enter the next command to run this system:

C#

The C# output could show warnings about asynchronous features now utilizing the await operator. You'll be able to ignore these.

Python

1. When prompted, enter 1 and observe the output, which ought to embrace the ID and attributes of every face detected.
2. View the detected_faces.jpg file that’s generated in the identical folder as your code file to see the annotated faces.

There are a number of further options out there inside the Face service, however following the Responsible AI Standard these are restricted behind a Restricted Entry coverage. These options embrace figuring out, verifying, and creating facial recognition fashions. To study extra and apply for entry, see the Limited Access for Azure AI Services.

For extra details about utilizing the Azure AI Imaginative and prescient service for face detection, see the Azure AI Vision documentation.

To study extra concerning the Face service, see the Face documentation.

Courtsy: Azure and Microsoft