FaceTrack Pro

Use the FaceTrack Pro module to recognize the face of a person of interest (e.g., a suspect or a missing child) or solve some other real-time and investigative analytics use cases according to the detailed descriptions in this chapter below.

Meet FaceTrack Pro

How it works

In practice, FaceTrack Pro deals with solving an open-set recognition problem. In this sort of issue, a vast set of candidates, Users, or Imposters, is compared with a much more narrow list of enrolled Users in search for the best match or making a “No match” (“Imposter”) prediction.

At input, a trained Convolution Neural Network (CNN) continuously analyzes every n-th frame in the incoming video stream to find out if there is a face in it regardless of angle and distance. In between, those faces, if any, are tracked.

Once a face is detected, the face bounding box goes to CNN-2, which is trained to extract feature vectors (descriptors) from it. The extracted descriptors are then compared to those of every person enrolled in a database to find the best (above the threshold) match, if any, thus making a "Match <person ID>" or “No match” (“Imposter”) prediction.

As compared to still photo recognition, face recognition in the video (FRiV) takes advantage of having multiple attempts (frames) to extract the best feature vector as long as the face stays within the camera view range.

The module generates two types of face recognition events:

• Match means that a face was detected in a video stream from a camera that was found similar to someone in the system database within a user-defined similarity score. The match's full name will show together with more personal data available in the system database.

Searching for a match,  the program iterates through all the person lists in the database with the Recognition mode option enabled and a similarity score defined.

• No match means a face was detected in a video stream from a camera yet not similar enough to anyone in the database. 

Computational capacity

The computational complexity of the module is estimated "High", which means that up to 100 % of the computational capacity of the processing core is used by the video analytics module (although the exact numbers may vary with video resolution and CPU parameters).  

Camera requirements

• Camera position: fixed or mobile (PTZ, mobile phone uploads, drones).
• Resolution: from 1280x720 (HD Ready) to QHD 2592x1944.
• Frames per second: 10 - 30 fps.
• Main/Secondary stream bitrate, up to: 8/1 MB/sec
• Codec: H.264 HP/MP/BP
• Net protocols: TCP/IP, ICMP, HTTP, HTTPS, RTP, RTSP, NTP, SNMP, IGMP, IPv4/v6
• Advanced features: DNR, BLC(HLC), WDR 120dB 

Camera mounting requirements

• Eye-to-eye distance: 50 px
• Lighting: natural or artificial, no backlight, no reflections.
• Image quality within the region of interest: over 500 px per meter
• Camera height: on the head level or slightly above (1.5-3 meters from the floor). This ensures the camera tilt is within +5°. 




• Social engineering: Use reception desks, ticket scanners, and digital signage to attract attention. This helps keep the camera pan and roll from -5° to +5° 

The pictures below are to illustrate good and bad camera positions.

Good	Bad

Export module settings

Export the module settings to a *.xml file, e.g., to pass it to qualified IREX personnel to find a solution to a problem with the module performance, should it happen. To do so:

1. Find the camera running the module in the User menu — Settings — Cameras, and click Video  analytics settings as shown:    

2. On the VIDEO ANALYTICS tab on the camera settings pane on the left, click the three-dot menu 



Export analytics configurations of the module — to export the module settings on the MAIN sub-tab.

Export analytics configurations of the model — to export the entire module configuration, including the rules, if any.

Enable FaceTrack Pro

1. Find the camera to enable the module in the User menu — Settings — Cameras section and click Video analytics settings as shown.    

2. On the <module_name> list, click the name of the module.
3. Slide right the On/Off slider 
4. Click Apply at the bottom of the pane.

How to find a suspect

Finding a suspect may be an important part of a post-incident investigation. If this is the case, use any of the previously described techniques depending on what is known about the person: name or other text attribute, photo or distinctive features.

However, it might be a real-time use-case as well. Then do:

1. Analyze which of the existing cameras fit for face recognition.
2. Enable FaceTrack Pro on those cameras and set it up.
3. Add the suspect to a personal list in the database keeping in mind the ethical considerations and grant access to the list to the user group tasked with finding the suspect.
4. Build an alarm monitor with Lists = that in item 3 and  Analytics modules — FaceTrack Pro — Match checkmarked, configure notifications..
5. Whenever an alarm comes, verify it before taking any action that might seem like an unfounded intrusion into someone's privacy.

How to find a missing child

Since a highly sensitive issue is touched upon, close attention should be paid to the correctness and updateness of information on the missing child list in the person database.

A missing child's personal data can only be taken from an institution that has enough authority to possess and maintain it. Such database might contain multiple entries, which can be imported into a person list in batch mode. Then take special care to keep it continuously updated. However, a better solution would be to contact the Platform representatives, asking us to build an interface to the source database through the system API.

In any other respect, the missing child use case does not differ much from that of a suspect in real-time.

How to manage biometric access

Regulated access e.g., to premises, can be implemented using the face recognition capabilities of the Platform and the interfaces it has with external devices, such as smart locks. To do so, create a person list holding the entries of those who have access and configure the connection to a smart lock. However, pay special attention to a similarity threshold value in order to keep the acceptable balance between false negative and false positive predictions.

Set up face recognition

1. If there is a GPU to be tasked with face recognition calculations, switch on the Run on GPU option to speed up those.

2. Edit the Region of interest,which defaults to the full frame.

To move a vertex, point the cursor onto it and drag and drop it.
To delete a vertex, double-click it.
To add a vertex onto an edge, double-click where the vertex should be.
To move the region as a whole, place the cursor inside it and drag-and-drop.

3. Assign a priority to the "No match" events according to need.
4. Select the  Highest accuracy or Minimum delay performance mode on the Identify with ... list.
5. Switch on/off the the Recognize small faces option to improve detection of faces of about 5% in size of the frame height and smaller.
   The option is not available with the Run on GPU switched on.
6. Switch off the Recognize appearance features whenever it is irrelevant thus saving the system computational resources.
7. Switch on the Limit minimum face size option to ignore faces smaller in size than the limit, set graphically by resizing the yellow rectangle that appears in the frame.


This will prevent e.g. a biometric access camera from making too early "Match" or "No match" decisions e.g. when the candidate is too far from smart lock.

8. Switch on the Re-recognize face option according to need and set a Re-recognition interval.

With the option switched on, face recognition calculations will repeat every Re-recognition interval seconds if the face stays in the frame, and new events generated according to the result.

9. Switch on the Limit face angles option to lower the recognition error rate.

With the option switched on, recognition calculations wil not be run with faces tilted or panned above the limits.

10. Click the Apply to start streaming the “Match <name>” and "No match Unknown>" events onto the Events screen of the Platform.

Tips to review face recognition events

Create an alarm monitor holding the suspect's "Match" events (if his or her identity is known) or configure the monitor by suspect's appearance features (otherwise) to shorten the response time to an incident of this kind. However, an alarm on this monitor still needs verification before taking any action that might infringe on someone's privacy and human rights.

To verify an alarm, click the alarm card on the alarm monitor to open the pane on the right, holding:

• Time and location tags of the event
• Similarity score
• Person appearance features (demographic group, glasses, beard, etc.)
• The link to the full personal entry in the database.

Re-estimate the relevance of these data to the wanted person under the current circumstances. Playback the event to verify it before responding to it, keeping in mind the ethical considerations.

Verify to be ethical

All the existing face recognition algorithms (as well as a human brain) show different accuracy rates for different demographic groups of gender, age, and skin color. The bias is difficult to avoid due to natural variations in training datasets.  Although AI scientists are doing their best to eliminate all traces of this bias, still any actor in the area of face recognition should remember it. On the other hand, an incorrect response to a false identification may have serious consequences as for human right violation, financial and reputational damage, etc.

So, verify the event as thoroughly as possible before responding to it, taking into close consideration every relevant aspect.

• Above demographic bias, there might be other uncontrolled reasons for recognition errors. The best way to estimate error probabilities in each particular Customer's environment is to collect a statistically significant amount of raw data (video) and pass it to the IREX team for measurements directly in an experiment.
• If in doubt, system logs are available to show who added a person to the database and when, and what the grounds were.
• Personal data can also be checked for being up-to-date to make sure the right to be forgotten is respected in each particular case.