AI workloads and considerations:

Features of common AI workloads

1. Features of anomaly detection

   1. identify fraud or failure
   2. use Anomaly Detector:
      1. embed time series data of all types
      2. select best algorithm
      3. deploy where you need (cloud or ioT edge)

2. Computer Vision workloads:

   1. image classification, object detection, semantic segmentation (different cars of different colors → Semantic segmentation provides the ability to classify individual pixels in an image depending on the object that they represent), image analysis (tags, summarize scenes), optical character recognition (OCR - text in images)

   2. use Computer Vision:

      1. analyze images and video, extract description, tags, objects.

   3. use Custom Vision:

      1. train custom image classification and object detection using own images.

   4. use Face:

      1. build face detection and face recognition.

   5. use Form Recognizer:

      1. extract info from scanned forms and invoices (’fatura’)

   6. NLP workloads

   7. analyse and interpret text in documents, emails, etc.

   8. interpret spoken language

   9. auto translate spoken or written phrases between languages

   10. interpret commands and determine appropriate actions

   11. use Language:

       1. access features for understanding and analyzing text, trainning language models that can understand spoken or text-based commands

   12. use Translator:

       1. translate text between more than 60 languages

   13. use Speech:

       1. use this service to recognize and synthesize speech, and to translate to spoken languages. Any speech to text or text to speech translations.

   14. use Azure Bot:

       1. use bot framework to create a bot and manage with Azure Bot Service - integrating back end services like language, connect to channels from web, mail, teams, etc.

   15. Knowledge mining workloads

       1. extract info from large volumes of unstructured data to create a searchable knowledge store
       2. use Azure Cognitive Search:
          1. search solutions that has tools for building indexes
          2. can use built-in AI capabilities of Azure Cognitive Services such as image processing, content extraction, NLP processing to perform mining documents
          3. makes it possible to index previously unsearchable documents and to extract insights from large amounts of data
          4. Cognitive Search can index JSON documents. JSON is also used to define index schemas, indexers, and data source objects.
          5. An indexer converts documents into JSON and forwards them to a search engine for indexing.
          6. f you set up a search index without including a skillset, you would still be able to query text content. Cognitive Search is used for full text search over indexes containing alphanumeric content.

   Guiding principles for responsible AI

   1. Fairness
      1. interpret model and quantify the extent to which a feature influences prediction
      2. all groups and races and genders involved
      3. Fairness involves evaluating and mitigating the bias introduced by the features of a model
   2. Reliability and safety
      1. not create harm in the world
      2. rigorous testing and management
      3. operate as they were originally designed for
      4. resist harmful manipulation
   3. Privacy and safety
      1. secure and respect privacy
      2. data contain personal details that must be kept private
      3. security on the new data after deploy
      4. control over collection, use and storage of data
   4. Inclusiveness
      1. empower people
      2. bring benefits to all parts of society
   5. Transparency
      1. AI should be understandable
      2. users should be fully aware of purpose and conditions of the system
      3. Transparency provides clarity regarding the purpose of AI solutions, the way they work, as well as their limitations.
   6. Accountability
      1. people should be accountable
      2. developers should work under governance frameworks and ethical standarts

   Fundamental principles of ML on Azure

   Common ML types

   1. Regression
      1. continuous value (price, sales)
      2. evaluation metric: r2 and RMSE
      3. multiple linear regression: features are independent of each other
   2. Classification
      1. determine class label, binary
      2. classify text as positive/negative
   3. Cluster
      1. labels by grouping similar info
      2. to deploy: select module named ‘assign new data to cluster’

Azure ML Studio

To start modelling, you need to create an experiment.

Compute:

• instances: dev workstation
• clusters: scalable virtual machines
• inference clusters: deployment targets for predictive services that use trained models
• attached compute: VM on databricks clusters

1. use Automated ML
   1. you can include python scripts;
   2. automatically tries multiple pre-processing techniques and model trainning algorithms in parallel;
   3. no DS programming knowledge needed;
   4. jobs: specify training script, compute target and Azure ML envirionment.
   5. for other to use: deploy the model to a web service

Steps:

Inference pipeline (AKS - Kubernetes):

• inference in real time;
• to access web service: authetication key (primary) and REST endpoint

1. use Azure ML Designer
   1. First step: create a pipeline ind ML designer
   2. canvas with drag and drop interface;
   3. each designer product is a pipeline you make on canvas
   4. Assets:
      1. pipelines: lets you reuse complex ML flows across projects
      2. components: encapsulates a task for tracking your ML experiment
      3. jobs: executes a task for tracking your ML experiment

Steps:

1. create workspace
2. create compute
3. create pipeline
4. create dataset
5. load data to canvas
6. apply transformations (clena missing, normalize, etc)
7. run pipeline and verify data
8. add train modules and split data
9. run pipeline
10. score and evaluate
11. create inference pipeline
12. deploy as a service

Features: characteristics of the entity for which you want to make a prediction;

Labels: is the quantity you want to train a model to predict (predicted values);

Features of computer vision workloads

Common types of computer vision solutions

Image classification → analyse an imagr and generate human readable phrase that can describe the image. The description can be used to suggest tags (building, tower).

Object detection → return bounding box coordinates (multilabel and multiclass) and tags (class label and probability). Can be used to detect brands, human faces, domain specific contents.

OCR → detect printed and handwritten text in images.