Data Mining the City

First theorized in 1956 by Guy Debord of the Letterist International, the concept of the “dérive” denotes a technique of mapping an urban environment's psychological dimension. The practice involves drifting semi-aimlessly subject to the city's suggestive ambiances. Led by Danil Nagy, Jim Stoddart, and Lorenzo Villagi of New York-based architecture firm "The Living," this workshop began with participants conducting psychogeographic “dérives” while wearing equipment for recording biosignatures, audio, and 360-degree video. These streams of data were used to create an intricate 3d reconstruction of the experience overlaid on a point cloud, using structure-from-motion algorithms in the open-source software COLMAP. A screenshot of one of these pointclouds is displayed below. Members divided into teams to investigate different lines of research. I was part of a team interested in the fact that moving objects, such as cars, people, dogs, or the like, are systematically overlooked by structure-from-motion, hence completely absent from the 3d reconstructions. We sought to reinsert these objects into the virtual model by using computer vision to locate and label such objects in the 2d frames of the video, then compare these with the orientation and depthmaps recovered by structure-from-motion. The resulting virtual environment was made interactive through virtual reality. This project proposes a new way, enabled by open-source technology, to make the dérives of the Situationists an experience that can be shared.

Model of a 5-minute walk through a park in downtown Toronto

Frames represent moving objects

Shane Burger experiments with the VR interface