Project

Welkom to the webpage about my (Floris De Smedt) PhD project. On this page I will describe what this project is about. On the other pages you can find some information about myself, my publications and some code I am authorized to release. If you have any questions or remarks about this project,feel free to contact me.

Pedestrian detection has endless possibilities in many application areas. An evident application domain is surveillance. The number of cameras in public places such as train stations, shopping malls and streets grows each year, with security in mind. Currently there are almost 300,000 surveillance cameras active in Belgium, with an increase of almost 10% in the last year. Note that these numbers do not include the cameras used in cars, mobile devices, ... A direct response in the cause of an incident, however,  requires manual observation of the video stream, which is in most cases economically infeasible. This means that these cameras are mostly used to capture evidence material after the incident. We believe however that an improvement and more efficient use of object detection, and more specifically pedestrian detection, can be of help as a pre-processing step for continuous analysis of these video streams.

Pedestrian detection

When we look into the domain of traffic safety, we see that more and more modern cars are provided with a system to detect possible collision with pedestrian, cyclists and other cars. But larger vehicles, such as trucks, still cause to many casualties which could be avoided with the proper technology. Similar systems could be applied for improving safety around agricultural vehicles when working off-road.

Also in e-health applications could pedestrian detection play an important role. Just one example is the detection of falling elderly people to automatically trigger an alarm. This kind of supporting technology allows the elderly people to live longer in their familiar environment.
Fall detection of elderly

This broad applicability of human detection, and object detection in general, makes this topic receive a lot of interest over the past decade. A number of promising algorithms have been developed over time, and improvements on both accuracy and speed are published each year. Most of these algorithms are primarily tested in lab surroundings though, and/or are only available in Matlab, which makes them difficult to apply in real-life applications. The two main properties that determine the usefulness of detection algorithms are the evaluation speed, and the detection accuracy. Evidently does the evaluation speed depend on the computation platform, e.g. modern computers with state-of-the-art processors allow to perform a multi-threaded evaluation
at high processing speed, but such systems are not practically feasible when pedestrian detection has to be performed on-board from a flying drone (a case study we worked out in the context of this PhD). The detection accuracy on the other hand is strongly dependent on the complexity of the images. Detecting pedestrians before a high contrasting wall is far more easy compared to detecting them from a driving car in the precense of fog.

In this PhD study we worked out some techniques to make state-of-the-art pedestrian detection techniques  useable  in such real-life applications. In the context fo this PhD we developed techniques that improve both the detection speed and teh detection accuracy. To improve the detection speed, we worked out two techniques. In the first one we focussed on parallelisation, which we exploited both on CPU (multi-threading) and on GPU (using the languages CUDA and OpenCL). By performing multiple calculatings at the same time, the detection approach can be drastically improved. Another technique to improve the detection speed is to make the search space smaller. A pedestrian detection algorithms can be roughly described as the search for pedestrians at all locations and multiple scales (this technique is commonly desribed as Sliding Window). By pruning a lot of these locations based on scene information, this search process can be performed faster, and hence we obtain higher processing speeds. Herefore we worked on two approaches. In the first we considered the use of temporal information. The idea is based on the fact that when we find a pedestrian at a cetain location, in the following frame we expect the pedestrian to be at a close to the same location, and thus we have only to search around this location for that pedestrian. Another technique is exploiting the knowledge that if pedestrians go futher away from the camera. they will appear smaller in the image.

To improve the detection accuracy, we worked out a very efficient technique to combine the detection results of multiple pedestrian detection techniques. For this combination we use a confidence value, that indicates how trustworthy a pedestrian detection algorithm is on its own, and a complementarity value, which measures the additional value if multiple detectors detect at the same location. Note that this latter value depends on the similarity of the detection approaches.

For pedestrian detection, a selection of these techniques can be integrated. In the context of this PhD we worked out two such applications. the first one focussed on accurately detecting pedestrians in the blind spot of a driving truck. A high challenge here is the strict real-time constraint. We solved this challenge by combining parallelisation on GPU and CPU as a hybrid implementation. By combining this with heavy search space reduction, we obtained a speed of over 500 detections per second. This allows to reach still very high framerates such as 25fps with as many as 20 pedestrians in the scene.

In the second application we created a system that allows UAVs to follow pedestrians atonomously based on the location camera images. For this task we implemented pedestrian detection. pedestrian tracking (following pedestrians over time) and slogic for steering the drone on an embedded system that was mounted on top of the drone.  This allowed to directly use the pedestrian's location as input to the steering logic of the UAV, and hence in instant response to  follow the pedestrian in the scene. For this work we received the best paper award at the Embedded Vision Workshop at CVPR in Boston this year (2015).
 
This research is performed at the EAVISE research group at Campus De Nayer - KULeuven. This research is in co-operation with the PSI - VISICS research group at ESAT K.U.Leuven.