Rijkswaterstaat is the government organization responsible for the design, construction, management and maintenance of the main infrastructure facilities in the Netherlands. Commissioned by Rijkswaterstaat’s Data Lab we developed this ‘artificial inspector’ as a demo; a self-learning system for the inspection of highways and waterways. The artificial inspector is an example of deep learning using neural networks. The artificial inspector has been trained to detect and classify numerous items on images.
The safety of infrastructure is partly determined by the presence of the right signs and markings. On national roads, these are, for example, hectometer indicators, traffic signs, information portals and lighting. On waterways traffic is directed by signs, buoys and beacons. With the help of the artificial inspector, a large part of the inspection of these objects is automated in the following way. Images are collected with cameras. Instead of using expensive professional cameras (which can be used to a limited extent due to their cost price), Rijkswaterstaat has opted to use standard smartphones. The smartphones are mounted on cars and ships. Our for this purpose specially developed Android app makes a recording every second and forwards it to the artificial inspector.
The artificial inspector is able to recognize objects on the images. A database with geographical information is then used to check whether the recognized object matches with what should have been found at that specific location. The artificial inspector can also determine that something is missing that should have been present according to the geo-information. For example, we are able to identify damaged or missing road signs, to detect damage to the road surface or crash barriers or to signal whether buoys are missing or incorrectly placed. If there is an omission, only then a Rijkswaterstaat inspector will be informed. This saves the authorities time and the public money.