File Chaos: Why the Construction Industry Needs a New Approach to Managing Reality Capture Data

Do not index

The construction industry has a reputation of being slow to adopt new things, often constrained by tight budget scrutiny. Despite this, it has undergone transformative shifts in how buildings and infrastructure are designed, built, and maintained. While it is slow to adopt the latest innovations, the adoption of mature, proven tools has been consistently strong.

Urban designers before CAD software existed. (Source: https://rarehistoricalphotos.com/life-before-autocad-1950-1980)

The digital revolution in construction began in the 1980s with the advent of Computer-Aided Design (CAD), which replaced hand-drawn blueprints as the standard format for creating building designs. In the 1990s, CAD evolved into 3D design, adding a new dimension to visualization and planning. Then came Building Information Modeling (BIM), which introduced additional layers like time and budget, to manage entire project lifecycles. The 2000s saw the rise of mobile communications and cloud computing, bringing BIM workflows from the office to the construction site and improving collaboration between on-site staff and office-based teams.

Continuing the steady progression of innovations, the next major leap was reality capture. Smaller and cheaper laser scanners, and the proliferation of drones in the early 2010s, made it possible to create virtual 3D replicas of construction sites. Suddenly, teams could monitor progress, compare designs to as-built structures, and develop BIM models for renovation projects – perhaps not easily, but in a much more convenient way than before.

Today, we’re witnessing the rise of robotics and AI, building on these earlier breakthroughs—particularly reality capture. It’s easy to imagine where this is headed: fully integrated and autonomous construction sites, where robots handle not just scanning and capturing reality but also parts—or even all—of the construction process itself.

The Exponential Growth of Reality Capture Data

Market analysts predict a 21% annual growth rate for the LiDAR market, signaling an ongoing explosion in reality capture data volumes. Our partners’ experiences reinforce this trend. For instance, Infrakit, one of our early collaborators, has observed annual data growth rates of close to 100%. This surge is fueled by behavioral shifts, with customers moving from monthly scans to multiple scans per month, and eventually to continuous scanning. On the technological side, cost reductions and smaller form factors are enabling more sensors to be deployed on construction sites, capturing higher-resolution data than ever before.

Consider the advancements in aerial LiDAR. In the early 2000s, sensors captured just one point per square meter (ppm). Today, modern systems achieve densities of 60 ppm. By 2030, we expect that number to rise to 200+ points per square meter. With this combination of behavioral and technological shifts, the growth in data volumes is not just linear—it’s exponential.

File Chaos

Now, what does this mean in practical terms? Are we ready for this future? From the perspective of data management, not at all! Reality capture data is still managed as individual files, with each file representing a single scan mission. These files undergo multiple processing steps—georeferencing, registering, filtering, classification, and file type conversions—often across different software platforms and by various service providers. Each step leaves behind intermediate files that accumulate over time. Finding what you need, combining parts of different files, and using the data, in general, is a nightmare. Due to the nature of the data, files can sometimes reach terabyte scale, and they are often stored on external hard drives and even shipped over physical mail. Yes, this still happens in the year 2024.

We call this the File Chaos. This complicated pipeline leaves behind a trail of files that might prove useful in the future. Reliance on external storage media means the risk of version conflicts and duplicate copies is all too real. By the end of a multi-year construction project, thousands of files are scattered across network shares, team members' computers, and external drives.

Paradigm Shift: LumiDB

There is a better way, and it’s called LumiDB. What we’re building is a paradigm shift away from the file-based method of storing and accessing data. Reality capture data is inherently related to a specific spatial location and a point in time, with many associated metadata properties. LumiDB is a database system that organizes the data by these properties and provides an API for querying it. With this approach, data can be stored in a unified repository and integrated with processing tools, visualization applications, and analysis algorithms—just as conventional databases are used in other contexts.

In contrast to proprietary solutions with closed ecosystems, LumiDB is not a walled garden. It’s designed to be open and flexible, supporting integration to any workflow via an API and data export in standard formats for use in any software. Whether you’re visualizing, analyzing, or training AI models, LumiDB ensures your data is always accessible and easy to use.

While our focus is on construction, reality capture data management challenges extend far beyond this industry. Mining, geospatial services, and robotics face similar issues as their datasets grow. LumiDB is built to scale across industries, making reality capture data easy and fun to use and accessible no matter the application.

We’re currently in closed alpha testing and working closely with industry partners to refine our product. If you’re interested in being part of the future where File Chaos will look like an ancient myth, we’d love to hear from you. Get in touch to learn more!