Crowd Analyser

Introduction

Functionality

To increase customer satisfaction, skiers are increasingly supplied with current information about the ski area on local information displays, web information systems and mobile apps. The current lift waiting times are useful information in order to decide which ski slope should be chosen, in order to avoid long lift waiting times, to estimate if there is enough time for a specific route, or if a stop at an alpine hut/restaurant would be the better option. Another advantage is that the use of this information leads to a more uniform load distribution in the ski area and thus an overall reduction of waiting times.

alternate text

Previously, you could only measure the lift capacity utilization but that allows no conclusion to the actual waiting time at the lift. With viasense - Crowd Analyser there is now a system for automatically capturing information about the number of waiting skiers. For this, the waiting area is monitored at a ski lift with a camera. The image data is processed by image analysis software which detects the degree of filling of a defined waiting area (see Figure 1). From this value the waiting time of the persons can be derived.

Example of a defined waiting area for calculation of the relative size of the filling degree area

Figure 1: Example of a defined waiting area for calculation of the relative size of the filling degree area

The recognition results are not affected due to slight camera movements (as it may occur at camera pole mounting and in case of strong wind). The detection of the degree of filling is possible even in rain and snow. Only in very extreme weather conditions, the recognition could fail due to poor visibility conditions. However, such situations are recognized by the acquisition of image contrast information at background objects (e.g. buildings, lift towers, etc.) and can be taken into account when displaying the recognition results.

For each ski lift one video camera is required (see Figure 2). By using a fish-eye camera (camera with a very wide angle lens) the entire waiting area can be captured. Image distortion caused by fisheye lens is not a problem for the evaluation. Also, by the large field of view of such a camera it is ensured that no individuals can be identified. Image data is not recorded and even do not have to be displayed on a monitor.

Example for mounting a camera at a lift station (the recommended installation height of a 180 degree fisheye camera is about 5-6 m)

Figure 2: Example for mounting a camera at a lift station (the recommended installation height of a 180 degree fisheye camera is about 5-6 m)

The resulting data can be transferred via a web service interface (REST) to a display system (scoreboard or web server for an information web portal). In addition, an e-mail notification can be automatically generated when a detected degree of filling is above or below a threshold (threshold with hysteresis) or when the connection to a camera is terminated.

Requirements & Installation

Requirements

An example of the system configuration is shown in Figure 3. A lift waiting area is monitored by a camera. The processing of the image data is accomplished via a standard PC. The result data can be sent through a REST web service interface to a display unit. Alerting via email is possible, if you can access an email server.

Example of the system configuration

Figure 3: Example of the system configuration

The system requirements are highly dependent on the performance requirements, such as whether you want on a computer running one or more instances of the monitoring software or what resolution have the cameras used. Running the application on a PC with too little computing power, decreases the number of processed frames per second (fps), and thus the detection rate. A too low computing power is given when mostly for the frame rate a value is displayed which is less than 10.

The application is optimized for an Intel processor.

PC-Requirements:

The following table is an example of the recommended computer configuration given for two different image resolutions:

Number of cameras Recommended configuration  
  Image resolution: 1290 x 480 Image resolution 1290 x 960
1 camera Intel Core 2 Processor 2,2 GHz, 1 GB RAM Intel Core 2 Processor 3 GHz, 2 GB RAM
2 cameras Intel Core 2 Processor 3 GHz, 2 GB RAM Intel Quad-core Processor 3,2 GHz, 2 GB RAM
3 cameras Intel Quad-core Processor 3,2 GHz, 2 GB RAM Not recommended

Table 1: Recommended configurations

The PC must be equipped with a DirectX 9 compatible graphic card or unit. (e.g., for a PC with the motherboard ASUStek P8B WS and Intel Xeon product family E3-12x5 a separate graphics card is not required). The software runs under the operating system Windows 8, Windows 7 or Windows Vista.

Network requirements:

  • Network Connection: 10/100 Mbit/s Ethernet card
  • Optionally, a mobile network connection through a UMTS / HSDPA router (e.g. UR5) can be established (for the transfer of the recognition results and optional email alerts).
  • Email server (optional for e-mail alerts)

Directly supported cameras:

  • Type: network cameras by AXIS and MOBOTIX (Color or Black & White Cameras)
  • Minimum frame rate: 10 frames per second

Optional connection to a video surveillance system:

  • GeViScope

Selection of Cameras: The main criteria for the selection of the cameras is the size of the viewing area (wide angle), the image resolution and frame rate. The best cameras are with a 180 degree fisheye lens and an image resolution of about one megapixel and a frame rate greater than or equal to 10 frames per second (e.g. L12 M25 MOBOTIX).

Software Licenses: For each camera an instance of the application has to be started. For each instance a separate software license is required.

Installation

To install, run the setup.bat program. Thereby the installation of four software packages will be started. First, the actual application Crowd Analyser is installed (see Figure 4).

_images/CrowdanalyserFig4.png

Figure 4: Installer of the application Crowd Analyser

In a second step the necessary video decoder will be installed through the “JRS filter Package”. In the intermediate steps, the default settings can always be chosen. During this installation also the Setup program from the “ffdshow” and Haali “Media Splitter” is executed.

_images/CrowdanalyserFig5a.png _images/CrowdanalyserFig5b.png _images/CrowdanalyserFig5c.png

Figure 5: Installation of the “JRS Filter Package” with the components “ffdshow” and “Haali Media Splitter”

To enable a connection to an AXIS camera, in a third step the “AXIS Media Control Embedded Installer” is called (see Figure 6).

_images/CrowdanalyserFig6.png

Figure 6: AXIS Media Control Embedded Installer for the connection to an AXIS camera

Finally, the installation program for the DirectX runtime components will be started. After approval of the license agreement, the installation can continue.

_images/CrowdanalyserFig7.png

User Interface

The program for the detection of the degree of filling of an observed waiting area has the name CrowdAnalyser and can be started by the CrowdAnalyser.exe file in the “bin” folder. The corresponding icon can be seen in Figure 7.

_images/CrowdanalyserIcon.png

Figure 7: Icon of the application Crowd Analyser

In a software license, in which the application can be started (for multiple cameras) several times, the start of the programs is through the links “CrowdAnalyser - Kamera1”, “CrowdAnalyser - Camera2”, etc., displayed on the desktop and in the program menu. This allows saving different settings for each camera.

_images/CrowdanalyserUserInterface.png

Figure 8: User Interface of the program Crowd Analyser

The user interface of the Crowd Analyser program is shown in Figure 8. Below the camera image, the time and the number of processed frames per second (frame rate) are displayed. If images are analyzed from a video file, icons for flow control (start, stop and pause) and the indicator for the frame number (starting from zero) are also shown beneath the video image.

In the lower picture image positions are marked in green, where waiting people are detected. The brighter the green area is, the greater the reliability of detection and the greater the perceived density of people. On the left side of the interface the fill rate [0, 100%] from a defined waiting (see chapter “Determination of the Waiting Area”) is displayed. The arrows on this level display indicate the upper and the lower threshold (threshold hysteresis) for the e-mail alarm. These thresholds can be changed by holding down the left mouse button. Exceeds the detected fill rate for the first time the upper threshold, an email notification will be generated. Thereafter, only after the fill rate drops again below the lower threshold, additional email notification will be triggered.

User-Specific Setting of the Window Layout

The arrangement of the pans (camera image, display for the degree of filling, ...) of the user interface can be changed. The docking properties of a specific dock-window can be set in the menu bar by clicking on “View -> Configure -> Configure DockWidgets” (see Figure 9).

_images/CrowdAnalyserWindowSettings1.png

Figure 9: Menu item for changing the docking properties of a dock-widget

Once you have repositioned the panes, the new arrangement can be saved under a particular name in the menu “View -> Perspectives” (see Figure 10).

_images/CrowdAnalyserWindowSettings2.png

Figure 10: Menu for loading and saving of the pane arrangement

Selection of the video source

The image data can be received either directly from a camera or from a GeViScope video surveillance system. For testing purposes, a video file can be processed.

Camera connection

To connect to a camera, the camera icon in the toolbar has to be clicked (see Figure 11). Then a dialog box appears as shown in Figure 12.

_images/CrowdanalyserCameraConnection1.png

Figure 11: Camera symbol in the tool bar

AXIS and MOBOTIX network cameras can be used. Depending on the camera access restriction, additionally to the IP address the username and password must be specified. If the “Connect on Program Start” option is turned on, at the next start time of the application the connection to the selected camera is automatically established.

_images/CrowdanalyserCameraConnection2.png

Figure 12: Dialog window for the selection of a camera

Connection to the video management system GeViScope

To receive image data from a GeViScope server, click the third icon in the toolbar (see Figure 13). Subsequently, a dialog box appears for the connection settings (see Figure 14).

_images/CrowdanalyserCameraConnection1.png

Figure 13: Tool bar icon for the connection to the GeViScope server

Once the IP address or the server name, user name and password were entered, the connection can be started by the “Connect” button. If the connection was successful, you can select a media channel (camera connection to the server). If the “Connect at startup” option is turned on, the connection to the server GeViScope will be automatically established on the next time of the application viasense -Crowd Analyser. After pressing the OK button, the connection data are saved and the pictures from the selected media channel will be received.

_images/CrowdanalyserGeViScope.png

Figure 14: Dialog window for the connection settings to the GeViScope server

Video file

For testing purposes, you can select a video file as the video source. The file dialog for selecting a video file can be opened either via the menu item “File-> Open” or on the toolbar by the file open symbol. Recently opened video files can be accessed through the menu item “File -> Open Recent Files” (see Figure 15b). The supported video formats are listed in the table below.

Codec / Container MPEG-1 MPEG-2 AVI WMV
MPEG-1 Yes      
MPEG-2   Yes    
MS MPEG-4     Yes  
DV1/2     Yes  
Uncompressed RGB32     Yes  
WMV 7       Yes
WMV 9       Yes
Xvid     Yes  
DivX 3,4,5,6     Yes  
GeoVision     Yes  
CinePak     Yes  
MotionJPEG     Yes  

Table 2: Supported video formats

Determination of the detection area

The application enables to restrict the detection area in the image to avoid false detections and to reduce the computational effort.

To get into labeling mode, click the icon in the toolbar. When you position the mouse pointer in the window with the video image, a yellow rectangle appears as a mouse icon. Once they hold the left mouse button and move the pointer, a green area (semi-transparent) indicate the labeled area. If the right mouse button is pressed, a selected area will be deleted. With the mouse wheel you can increase and decrease the size of the drawing tool (yellow rectangle).

Determination of the waiting area

The waiting area is that area which is used for calculation of the degree of filling. The waiting area should only be part of the detection area (see chapter above). To avoid false detection, any areas should be excluded where people are covered by background objects (e.g. overhead displays).

To get into this selection mode, click the icon in the toolbar. The area can be labeled if the right mouse button is pressed.

_images/CrowdanalyserFig18.png

Figure 15: Labeled waiting area which is required for the calculation of the filling degree value

Determination of the observation time

The monitoring of the waiting area can be time-limited. With the toolbar icon you can open the dialog box for setting the observation period. In the dialog box, a time bar is located for each day of the week. The blue area in this time bar indicates when monitoring is enabled. This marker can be changed by keeping the left mouse button pressed while moving the cursor over the time bar.

_images/CrowdanalyserFig20.png

Figure 16: Dialog window for the determination of the observation time

For individual days (e.g. holidays, vacation days), you can specify a specific setting of the observation time. For that you have to click the button “Exceptions ...” (see Figure 21) and the corresponding dialog box appears (see Figure 22). The settings are carried out in the following steps: 1. In a first step, you label the days in the calendar (width the left mouse button pressed) for which you want to set the observation time. The selected days are listed in the text box which lying in the middle. Deleting individual entries is possible by reselecting the appropriate day in the calendar or in the list and by clicking the “Cancel Selection” button. 2. Next, you mark the observation period for the selected days in the time bar. 3. At last you can confirm the settings by clicking the “Save selection”. If you want a different observation period enter for one or several days, you have to repeat the above steps.

_images/CrowdanalyserFig21.png

Figure 17: Button for opening the dialog window for the definition of the observation time at exception days

_images/CrowdanalyserFig22.png

Figure 18: Dialog window for the definition of the observation time at exception days

Parameter settings

There are parameters settings, which affect the detection of the filling level, the display of recognition results, the web service interface, and the e-mail alert. These settings are saved in a file with the extension ”. Par”. The parameters can be changed in the application. It is not necessary to open the parameter file directly.

At program startup, the parameter file “CrowdAnalyser.par” (if multiple cameras are supported: CrowdAnalyser1.par, CrowdAnalyser2.par, etc.) is loaded. After a video source is specified the application automatically attempts to load a parameter file with the name of the video source (e.g. camera1.par or 10.8.232.170.par). A specific parameter file can be loaded by the toolbar icon.

The dialog box for the parameter settings can be opened with the icon. All changes of the parameter settings are active immediately. If you close the dialog box using the “Save” button, the settings are stored in the parameter file. Thereby you have the possibility to set the name of the parameter file. Parameter settings that are made specifically for a particular video source should be saved with the name of the video source. Thus the correct parameter file is loaded if an automatic restart of the application occurs (e.g. after a power failure).

Parameter settings for the image preprocessing

In tab “Preprocess Settings” you can see all parameters which influence all image processing steps and the display of the images.

_images/CrowdanalyserFig24.png

Figure 19: Parameter settings for the image preprocessing

The following settings are possible:

  • Color Inverting: For the camera images the color channels may be interchanged. This prevents that persons can be identified. This privacy protection action might be necessary, if the local image resolution (depending on the camera angle ) is very large and the image data is displayed on a monitor
  • Image Rotation: If the image orientation cannot be set at the camera, it is possible to rotate the camera images according to the specified rotation angle [0-359 degrees].
  • Scale Bottom Left, Right bottom Scale, Scale Top Left, Top Right Scale: Because of the camera angle, image areas with the same size in different image positions correspond to different waiting area sizes. Therefore, for the calculation of the degree of filling, local image areas will be weighted in accordance with these scaling values. The scale value of an image position is calculated by interpolation of these parameter values.

Parameter settings for motion detection

In tab “Movement Analysis Settings” the parameter can be changed which are required for the motion detection and thus for the calculation of the degree of filling.

_images/CrowdanalyserFig25.png

Figure 20: Tab for the parameter settings of the motion detection

In order to detect motion in the picture, distinctive image points (e.g. vertices) will be tracked. Waiting people are supposed to be there where movement trajectories end in the picture and where they fall below a minimum speed. For the calculation of the degree of filling the image is divided into patches (e.g. 16x16 pixels). An image patch is counted as filled if a minimum number (Cell Count Threshold) of trajectories end in this patch.

The following settings are possible: - Cell Count Threshold: If the number of point trajectories in an image patch is higher or equal this threshold value then the image patch will be counted for the calculation of the degree of filling. - Filling Degree Factor: The displayed value for the filling degree is equal to the detected value multiplied by this factor. - Grid Height, Grid Width: Image patch size.

Parameter settings for the web service interface

In the tab “Web Service Interface” the parameters of the REST web service interface can be changed.

_images/CrowdanalyserFig26.png

Figure 21: Tab for the parameter settings of the motion detection

The following settings are possible: - Send Interval [sec]: In the given time interval [seconds] the captured filling degree values are sent to a REST Web server. - Place: Name of the observation area (e.g. name of the ski lift). - Max Crowd Size: Estimated number of people which can be at maximum in the waiting area. This value will be sent to the REST web service (optional parameter). - URL: URL of the REST web server.

Contact & Support Information

Contact

Helmut Neuschmied
JOANNEUM RESEARCH Forschungsgesellschaft mbH
DIGITAL - Institute for Information- and Communication Technologies
Steyrergasse 17
A-8020 Graz
AUSTRIA
phone: +43 316 876 1159
fax: +43 316 876 1191

Licensing

For each camera an instance of the application has to be started. For each instance a separate software license is required.