Samples

These are some samples using Asilla SDK:
Sample 1: Receive alert of abnormal action from Asilla SDK client

In this sample, we will guide you to create a program to receive alerts of abnormal action from the Asilla SDK client and send notifications to the Telegram app.

Step 1: Install Asilla SDK client package
Install Asilla SDK client package as instructed here. In this example, we choose to install on a Jetson NX device.

Step 2: Connect Asilla SDK client to an IP camera
Login to Jetson NX device that has been installed Asilla SDK client and run below command:
asdk set_camera_url 1 rtsp://192.168.0.101/media/video1

"rtsp://192.168.0.101/media/video1" is your camera url of your IP camera.

Step 3: Create a Bot on Telegram
  • Open Telegram app.
  • Chat with BotFather with command /newbot and follow by BotFather’s guide.
  • After created a Bot, you can get your BOT HTTP API Token, see below image.
  • On Telegram app, find your Bot by name and chat some message.
  • Visit this URL https://api.telegram.org/botXXX:YYYYY/getUpdates (replace the XXX:YYYYY with your BOT HTTP API Token you just got from the Telegram B otFather) and get your Chat ID.
After this step, we will have BOT HTTP API and Chat ID.
See more about Telegram Bot here.

Step 4: Create a webhook to receive alert
Create a Python script name app.py as below.
    1
    2
    3
    4
    5
    6
    7
    8
    9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   42
   43
   44
   45
   46
   47
   48
   49
   50
   51
   52
   53
   54
   55
   56
   57
   58
   59
   60
   61
   62
   63
   64
   65
   66
   67
   68
   69
   70
   71
   72
   73
   74
   75
   76
   77
   78
   79
   80
   81
   82
   83
   84
   85
   86
   87
   88
   89
   90
   91
   92
   # -*- coding: utf-8 -*- 
   from flask import Flask, escape, request, jsonify
   from datetime import datetime, timedelta
   from io import BytesIO
   import os.path
   import json
   import base64
   import requests
   
   app = Flask(__name__)
   
   STAY_ACTION = -2
   ABNORMAL_ACTION = -1
   STAGGERING_ACTION = 4
   FIGHTING_ACTION = 5
   FALLING_ACTION = 6
   INTRUSION_ACTION = 8
   UNDERSKIRT_ACTION = 9
   THREATENING_ACTION = 14
   GROUPING_UP_ACTION = 19
   
   DISPLAY_ACTIONS = {
     ABNORMAL_ACTION   : "abnormal",
     STAY_ACTION       : "stay",
     STAGGERING_ACTION : "staggering",
     FIGHTING_ACTION   : "fighting",
     FALLING_ACTION    : "falling",
     INTRUSION_ACTION  : "intrusion",
     UNDERSKIRT_ACTION : "underskirt",
     THREATENING_ACTION: "threatening",
     GROUPING_UP_ACTION: "grouping"
   }
   
   @app.route('/receive_abnormal_alert', methods = ['POST'])
   def main():
       # Replace the XXX:YYYYY with your BOT HTTP API Token
       TELEGRAM_URL = "https://api.telegram.org/botXXX:YYYYY/sendPhoto"
       TELEGRAM_URL_GIF = "https://api.telegram.org/botXXX:YYYYY/sendAnimation"
   
       # Replace the ZZZZZZZ with your CHAT ID
       CHAT_ID = "ZZZZZZZ"
   
       bodyInfo = request.json
   
       time_format = '%Y-%m-%dT%H:%M:%S'
       eventTime = str(bodyInfo['eventTime'])
       eventTime = datetime.strptime(eventTime.split('.')[0], time_format)
       
       msg = ("Recognition: " + parse_recognition(bodyInfo['recognition']) + "\n" 
           "Location: " + bodyInfo['location'] + "\n" 
           "AttachedImageType: " + bodyInfo['attachedImageType'] + "\n" 
           "EventTime: " + eventTime.strftime('%Y-%m-%d %H:%M:%S') + "\n" )
       
       if "device" in bodyInfo:
           msg = msg + "Device: " + str(bodyInfo["device"]) + "\n"
       
       send_info = {'chat_id' : CHAT_ID,
           'caption' : msg
       }
       
       upload_files = bodyInfo['attachedImages']
       file_content = base64.b64decode(upload_files[0])
       
       files = {
           'photo': ('Image_' + bodyInfo['eventTime'] + "." + bodyInfo['attachedImageType'], BytesIO(file_content))
       }
       
       url = TELEGRAM_URL
       if bodyInfo['attachedImageType'].lower() == "gif" :
           url = TELEGRAM_URL_GIF
           files = {
               'animation': ('Image_' + bodyInfo['eventTime'] + "." + bodyInfo['attachedImageType'], BytesIO(file_content))
           }
       
       r = requests.post(url, data=send_info, files=files)
       
       res = {'status': 'OK', 'message': 'Sent alert to Telegram.'}
       return jsonify(res)
   
   def parse_recognition(recognition):
       if isinstance(recognition, list):
           labels = ""
           for item in recognition:
               action_name = DISPLAY_ACTIONS.get(int(item["type"]), str(item["type"]))
               labels += ", " + item["label"] + "(" + action_name + ")"
           labels = labels.replace(", ", "", 1)
           return labels
       else:
           return recognition
   
   if __name__ == '__main__':
       app.run(debug=True, host='0.0.0.0', port=5050) #run app in debug mode on port 5050
   

Step 5: Run webhook in flask
$ python3 app.py 
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://0.0.0.0:5050/ (Press CTRL+C to quit)
 * Restarting with fsevents reloader
 * Debugger is active!
 * Debugger PIN: 834-390-594

Your webhook endpoint will be as below:
http://{WEBHOOK_IP_ADDRESS}:5050/receive_abnormal_alert

{WEBHOOK_IP_ADDRESS} is IP Address of your machine you're running the webhook.

Step 6: Set webhook to Asilla SDK Client
Use curl command to run below command:
curl -X POST http://{SDK_IP_ADDRESS}:5000/setCommonCamConfig \
-H "Content-Type: application/json" \
-d '{"config_info": {"alert_api": "http://{WEBHOOK_IP_ADDRESS}:5050/receive_abnormal_alert"}}'


{SDK_IP_ADDRESS} is IP Address of your machine you have installed Asilla SDK Client.

Step 7: Enable sending alert
Use curl command to run below command:
curl -X POST http://{SDK_IP_ADDRESS}:5000/setSpecificCamConfig \
-H "Content-Type: application/json" \
-d '{"cam_id": {CAMERA_ID}, "config_info": {"alert_enable": true}}'


{SDK_IP_ADDRESS} is IP Address of your machine you have installed Asilla SDK Client.
{CAMERA_ID} is ID of Camera that you have configured.

Step 8: Perform some abnormal action and get the alert on Telegram app.



Sample 2: Receive alert of line-cross from Asilla SDK client

In this sample, we will guide you to create a program to receive alerts of line-cross from the Asilla SDK client and send notifications to the Telegram app.

To install Asilla SDK client package and setting up IP camera connection, please see Step 1, Step 2 at Sample 1.

To configure line-cross, please follow the below steps.

Step 1: Access web GUI of Line-Cross application
Open below URL by browser:
http://<IP address of the device>:5000/line_crossing

Step 2: Select the camera
Select the camera which you want to run the Line-Cross application with, you can simply select the camera in the list of Web GUI.

Step 3: Capture an image from the camera
After selecting the camera, press the "Capture camera" button in order to define the line.

Step 4: Define the line
Draw a line just by clicking on two terminals of the line to be defined on the captured image. The system counts individuals who cross this line.

Step 5: Apply settings
Press the "Apply" button, once you pressed the "Apply" button, the Line-Cross application automatically starts for the camera.

Step 6: Create a Bot on Telegram
If you have created a Telegram bot, please skip this step. If you want to create a new Telegram bot, please see Step 3 of Sample 1.

Step 7: Create a webhook to receive alert
Create a Python script name app.py as below.

    1
    2
    3
    4
    5
    6
    7
    8
    9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   42
   43
   44
   45
   46
   47
   48
   # -*- coding: utf-8 -*-
   from flask import Flask, escape, request, jsonify
   from datetime import datetime, timedelta
   from io import BytesIO
   import os.path
   import json
   import base64
   import requests
   
   app = Flask(__name__)
   
   @app.route('/receive_line_cross_alert', methods=['POST'])
   def main():
       # Replace the XXX:YYYYY with your BOT HTTP API Token
       TELEGRAM_URL_MESSAGE = "https://api.telegram.org/botXXX:YYYYY/sendMessage"
       # Replace the ZZZZZZZ with your CHAT ID
       CHAT_ID = "ZZZZZZZ"
   
       bodyInfo = request.json
       
       time_format = '%Y-%m-%dT%H:%M:%S'
       eventTime = str(bodyInfo['eventTime'])
       eventTime = datetime.strptime(eventTime.split('.')[0], time_format)
       msg = "Location: " + str(bodyInfo['location']) + "\n"
       if "device" in bodyInfo:
           msg += "Device: " + str(bodyInfo["device"]) + "\n"
       line_cross = bodyInfo['line_cross']
       if len(line_cross) != 0:
           line_cross = bodyInfo['line_cross']
           from_lc = datetime.strptime(line_cross['from'].split('.')[0], time_format)
           to_lc = datetime.strptime(line_cross['to'].split('.')[0], time_format)
           msg += "======Line crossing status======"+ "\n"
           msg += ("In: " + str(line_cross['in']) + "\n"
                   "Out: " + str(line_cross['out']) + "\n"
                   "From: " + str(from_lc.strftime('%Y-%m-%d %H:%M:%S')) + "\n"
                   "To: " + str(to_lc.strftime('%Y-%m-%d %H:%M:%S')) + "\n"
                  )
           send_info = {'chat_id': CHAT_ID,
                        'text': msg
                        }
           url = TELEGRAM_URL_MESSAGE
           requests.post(url, data=send_info)
           res = {'status': 'OK', 'message': 'Sent alert to Telegram.'}
           return jsonify(res)
       return
   
   if __name__ == '__main__':
       app.run(debug=True, host='0.0.0.0', port=5050)  # run app in debug mode on port 5050
   

Step 9: Run webhook in flask
$ python3 app.py 
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:5050/ (Press CTRL+C to quit)
* Restarting with fsevents reloader
* Debugger is active!
* Debugger PIN: 834-390-594

Your weekhook endpoint will be as below:
http://{WEBHOOK_IP_ADDRESS}:5050/receive_line_cross_alert

{WEBHOOK_IP_ADDRESS} is IP Address of your machine you're running the webhook.

Step 10: Set webhook to Asilla SDK Client
Use curl command to run below command:
curl -X POST http://{SDK_IP_ADDRESS}:5000/setCommonCamConfig \
-H "Content-Type: application/json" \
-d '{"config_info": {"alert_api": "http://{WEBHOOK_IP_ADDRESS}:5050/receive_line_cross_alert"}}'


{SDK_IP_ADDRESS} is IP Address of your machine you have installed Asilla SDK Client.

Step 11: Enable sending alert
Use curl command to run below command:
curl -X POST http://{SDK_IP_ADDRESS}:5000/setSpecificCamConfig \
-H "Content-Type: application/json" \
-d '{"cam_id": {CAMERA_ID}, "config_info": {"alert_enable": true}}'


{SDK_IP_ADDRESS} is IP Address of your machine you have installed Asilla SDK Client.
{CAMERA_ID} is ID of Camera that you have configured.

Step 12: Perform going over the line that is configured and get the alert on Telegram app
By default, Asilla SDK will send Line-Cross alert every 60 seconds. You can change the frequency by using this API to configure lc_count_frequency parameter.

To stop the Line-Cross function for a certain camera just by running the following command.
asdk disable_lc [cam_id]



Sample 3: Train abnormal behavior recognition model

There are few steps to follow in order to train the abnormal behavior recognition model. In this sample, we will guide you through those steps.

Basically, this application has three modes:
  • Collecting mode: Collects action samples / data for training
  • Training mode: Trains the model with collected action data
  • Running mode: Runs with the trained model
When the application is executed from the scratch, it runs in the order illustrated in the image below:

Before the execution, you can check and change the configuration settings as you wish (The application comes with the default configuration settings set by Asilla).

Step 1: Check and set minimum and maximum samples needed for training.

You can do this for each camera separately. Use the API to get and set the values for minimum and maximum samples needed for training.

To get the values of the parameters, refer to the Get specific camera config endpoint in the API.
To set the values, refer to the Set specific camera config endpoint in the API.

Set the attribute values (train_max_samples, train_min_samples) as you wish, but we recommend you to keep them within the range as explained in the table below,
train_min_samples train_max_samples
Edge devices 5,000 20,000
GPU machine (PC / Server) 10,000 50,000

We have set a default value of one hour for the minimum duration for collecting data which is included in checking conditions for switching the application from collecting mode to training mode. In our first release, asillasdk_client_v1.0, it's not possible to access that attribute through the API.

Step 2: Run training.

You can simply run the command for training at Camera ID 1 as below command:
asdk train 1
You can train for each camera separately or for multiple cameras at once. To train Camera 1 and Camera 2 together, execute the following command:
asdk train 1,2

After successfully running above commands, the system will start in the collecting mode, and keep collecting data until the conditions are satisfied for switching to the training mode.

There are two conditions for switching the system from collecting to training. If either of them is satisfied, then the system will automatically switch to training mode.
No Condition Switch
1 Samples > train_min_samples and Duration > minimum duration for collecting data(one hour) Yes
2 Samples > train_max_samples Yes (regardless of minimum duration for collecting data)

After the training process is done, the system will automatically switch to the running model. If you haven’t executed the training command (asdk train) along with the starting command (asdk run_client), then the system will keep collecting data until you execute the training command later on. We discourage that method because it may collect more than necessary amounts of data and you may need to manually handle the data before training keeping them in the recommended range.