A new integrated reality capture solution uses computer vision, AI and a proprietary handheld camera with GPS, to capture infrastructure at scale
Over the last ten years the AEC industry has been inundated with a plethora of reality capture technologies, from affordable laser scanners and SLAM devices to aerial and terrestrial photogrammetry – even iPhones with LiDAR.
Last month saw the introduction of a novel device and platform from Looq AI, a startup that claims to offer a ‘one-stop solution’ for surveyors, engineers, contractors, and asset owners to capture infrastructure, in minutes, with ‘survey-grade accuracy’.
Looq’s initial focus is on the utility sector, but there are also plans to expand into other industries. By making the technology incredibly easy to use, and fast at capturing real world assets, the aim is to make survey ‘integrated and continuous’ across engineering and construction.
The Looq platform is powered by what the company describes as a novel and fundamental computer vision and AI technology. The device is a proprietary handheld ‘Q’ camera, which combines four high-resolution cameras with ‘survey-grade’ GPS and an AI processor.
Data from the camera is uploaded to the cloud and automatically processed with Looq’s AI-based image-to-model software to create a high precision, sub-centimetre, and geo-referenced intelligent 3D ‘digital twin’, along with orthographic maps and AI semantic information.
Through the platform, project teams can then interact with the ‘digital twins’ and AI generated information to complete a multitude of engineering, surveying, analysis, and coordination tasks. There’s a web-based application for visualisation, analysis, and collaboration. Data can also be exported to a variety of formats.
AEC Magazine’s Greg Corke caught up with Dominique Meyer, PhD., CEO at Looq AI, to delve deeper into the technology.
Greg Corke: One of the key things about Looq that piqued my interest is that your AI technology includes ‘algorithmic processing that extracts critical assets, semantic and geometric features’. We’d love to understand more about that.
Dominique Meyer: At Looq, we’ve built a fundamental AI and computer vision technology that allows us to very quickly compute geometrically correct geometry with photometric accuracy just from images.
The big question is, how do you guarantee that geometric accuracy and quality? And that is exactly where our AI comes in. We have intelligence in terms of what the scene sees, what those objects are. That allows us to extract different feature classes, feature assets, and feature intelligences to both guarantee that geometric accuracy, as well as to enable downstream applications math.
Greg Corke: So, you’re recognising features, you’re recognising that a telegraph pole is a telegraph pole, and by recognising the objects, that can help define how your AI processes the photogrammetry, to try and extract the right kind of object data from it?
Dominique Meyer: Exactly. It’s a very unique way of getting us to that capability. And we strive to uphold that quality. In terms of actual object detections and semantic segmentations, we go quite far.
And this goes beyond maybe just saying, ‘hey, there’s a pole here’. It goes as far as saying, ‘hey this pole now has a set of features that may indicate that its material is ‘x’, or that its state of structural health is indicative based on a number of heuristics that are AI deterministic. Is it geometric tilting, is it colourisation from, for example, rotting or rusting?’ All these characteristics can feed into what we call asset intelligence, which we’re very excited by.
Find this article plus many more in the March / April 2024 Edition of AEC Magazine
👉 Subscribe FREE here 👈
Greg Corke: Can you give more examples of the objects that it can recognise, or is trying to recognise at the moment?
Dominique Meyer: Today, we support a multitude of object classes. What you’ll see in standard urban and rural environments, those include streetlights, power poles, power lines, different types of curbs, ground vegetation, buildings, assets, any kinds of vehicles and people (that we will automatically remove), all the way into very specific engineering use cases.
We’re very focused on supporting the distribution, EV charging assets, utility environments, and that’s where we’re looking at conductors, we’re looking at insulators, fuses and all those things. And those provide real value towards the engineering companies that we support.
Greg Corke: Because your system is aware of the geometric shapes of these objects, when it’s interpreting data for something like a curb, would it know that there’s a straight line and maybe a curved edge, so it would process the data in that way, rather than generating a complex mesh?
Dominique Meyer: We do a level of intelligence and how we apply statistical filtering and statistical robustness to our algorithms. And those go beyond just ‘object fitting’, those go beyond just saying, ‘hey, you know, we’re going to make up that there is a curb here’. We drive every single bit of our output data from matching from intelligence on those algorithms by using very fundamental statistical methods and AI to get there.
Our output is a combination of different assets. A point cloud is one of them; statistical and object level models is another one. But we really want to make sure that the different use cases are supported.
I think the curb you touched on is a very important one, specifically for any topographic surveys for any urban environments that are designed, because certain environments you’re looking at need to be ADA compliant [standards for accessible design], so tolerances on a millimetre scale, accuracy, geometric consistency, top of curb, bottom of curb, back of curb. All those things are things that are important to the survey world, and we care about upholding those accuracies across all of our asset classes.
Greg Corke: Continuing the curb example, would the object comprise planar surfaces defined by four points, or would it be a highly complex mesh, which would essentially represent the same thing, but obviously with a much more detailed data set?
Dominique Meyer: I’m going to say we support both. And the reason for that is because certain applications require fine levels of detail, and that means that if you’re going from the pour of a concrete edge of a curb into a plate, sometimes there’s a four- or five-millimetre jump. That cannot be modelled as a planar surface because you would not meet those geometric accuracies. But there are many, many use cases where you don’t need millimetre accuracy. And that’s where we do support a lower-level interpolation as well, that’s really enabling certain downstream applications, to make sure that there is integration into, for example, Civil 3D, and other pieces of software that are absolutely necessary in the industry.
Today, we see a lot of players working on increasing that resolution and capability of handling big data, and that’s one thing I think Looq is focused on, how do we foster that adoption towards Big Data, while also making sure [we enable] certain workflows that don’t necessarily support, terabytes and terabytes of geometric data?
Greg Corke: When the data is in the Looq AI platform and analysed, you would always store the highest resolution model in there, and then you could stream out different level of detail models, as required, for each use case.
Dominique Meyer: Exactly. One workflow within our platform is to create topographic survey reports. Another is exporting and integrating it into a 3D surface, for example, into some of the Autodesk tools, and that’s where we automatically do some down sampling to make sure we support integration and the use cases where you don’t need that full high density.
Greg Corke: In general, how does accuracy compare to things like laser scanning, iPhone LiDAR, handheld SLAM and other photogrammetry technologies?
Dominique Meyer: Accuracy is very important in our industry. We care about supporting relative accuracy, so point to point distance measurements, and we care about supporting global accuracies.
In our case, taking the photogrammetry and computer vision-only route, without supporting lasers in our core operating pipeline, has meant that we had to rethink how we guarantee those accuracies at scale.
Much of your audience may be familiar with the difficulties of large-scale photogrammetry and alignments. Often you see doubling, often you see noise characteristics. All of those have been addressed fundamentally at Looq and that’s how we guarantee that the data quality we put out is really up there.
If you look at iPhone LiDAR systems, they’ve made huge advancements. I think the cell phone industry has pushed the frontiers on what’s possible in terms of 2D and 3D sensing or imaging at a price point that is incredible. But you also have to remember that with a phone-based camera and LIDAR system you will have a limited field of view. And with Looq we are focusing on capturing a lot of information quickly.
If you look at an EV charging site that has been built, and you’re doing a pass down the road and come back the other side because of traffic danger or risk, within two, three minutes, you’ve got it captured.
With an iPhone you’ve got to capture around, you’ve got to scan up and down, go into different areas, and that creates a burden on the capturer. You need a very rigorous technique, and training for your field technicians to be able to get that. At Looq, we’re focused on making data capture simple while guaranteeing the data quality and integrity.
Going a little bit into the other LiDAR SLAM systems, this is where, we’ve seen, sometimes issues on the global drift and that’s an inherently difficult thing to do.
When you’re aligning point clouds over distances, or continuous sampling of those LIDAR systems, you’ve got to have some geometric or motion estimation that allows you to align that.
With Looq’s computer vision algorithms, we’re able to have incredibly robust geometric priors and matching constraints that allow us to guarantee a space and ultimately reconstruct it locally and globally accurate, and that’s how we really hit those higher accuracies in the system and carry the trust of the industry.
Greg Corke: Can you explain more about your Looq AI capture device? You’ve got a GPS sensor and four cameras, but where are they to give you a much wider field of view?
Dominique Meyer: We have one camera on each side, one on front and one on top, one on the left and right. That gives us a near 360 coverage, ensuring that you can see down below for ground capture, for topographic survey, and then also overhead for any overhead utilities and assets.
Greg Corke: And everything is being geo-located with the survey grade GPS receiver?
Dominique Meyer: What we have developed is a way to combine GPS accuracy, the GPS localisation, with incredibly robust geometric priors of our computer vision algorithm to get those accuracies that have previously just not been able to be attained at large scales.
Greg Corke: So, you could just turn up on site, and start scanning and it would be geo located accurately, without having to set up survey control points? It does all that through the GPS, where you are, the orientation, everything, and then the data is created and referenced relative to that?
Dominique Meyer: Exactly. We do have the opportunity for very specific use cases – and I want to list two here – to be able to import external control.
A lot of assets, specifically in city topographic surveys, require alignments to city monuments. Those are local coordinate systems set up by the city or utility to make sure that the deliverables are all consistently in that same coordinate system at a certain epoch. That is one use case where we do allow our customers to bring in that survey control, those survey control points, to be able to turn run a ground control alignment. Another one, of course, is in areas where you have limited GPS quality.
Greg Corke: I’ve noticed that there’s an AI processor on the device and then there’s more AI processing in the cloud. What does the on-device AI processor do?
Dominique Meyer: We’re doing hundreds of megapixels per second. And what that means is that you’re capturing a lot of data. And as you want to bring that data back to the cloud, you’ve got to, first of all, compress it and you’ve got to also decide how to best capture it. And that’s where our AI processor comes in. It identifies, based on your motion, based on the scene you’re capturing, how and when to best capture the data to ensure the best possible outcome in terms of the reconstruction, and then of course, the data efficiency.
As you know, being familiar with a lot of LIDAR systems, you often spend hours and hours capturing data, leaving you with hundreds and hundreds of gigabytes of point clouds. Manipulating and operating those in the cloud or locally is difficult. At Looq, what we’ve built is actually a proprietary way of also putting the data together into a format that is incredibly data efficient. And that means that pushing the data into the cloud becomes easy, it means that handling those files for processing becomes a lot easier. And that’s also a very unique way of thinking about how do we make that workflow even easier; how do we make that upload process even easier? So really encapsulating the most out of that and all that is powered by our on-camera AI processor.
Greg Corke: So, if you had multiple photos, which were all the same, or of a similar orientation, it wouldn’t upload all of those photos, it would only upload the ones that are relevant for the reconstruction?
Dominique Meyer: Exactly. That’s the basis for data efficiency.
Greg Corke: How big are the datasets that are uploaded to the cloud?
Dominique Meyer: We’re talking gigabytes, where you might have seen tens of gigabytes, hundreds of gigabytes in the LiDAR world.
Greg Corke: While still on site, how do you ensure you have good coverage, that you have captured all the data you need, so you don’t have to return later and fill in the gaps?
Dominique Meyer: That’s part of our ‘claim to fame’, getting someone new to using Looq up and running can be done in 5-10 minutes. So, being able to operate a simple capture system, powered by complex technology, means that now we can reduce that burden of integration and adoption and training for a lot of these engineering and survey companies.
The feedback we do provide on the system allows for a very quick way of saying ‘hey, the capture data path is good, the data quality we expect is good, the cameras were all working, and the data was successfully stored. Those feedback mechanisms ensure that all our customers happily capture at scale without having to worry that ‘hey, I didn’t see that, or I didn’t capture that, or we are missing data because there was a system crash’. All these things are taken care of by the very, very robust underlying architecture of our technology.
You have a live preview of the images being captured [displayed on the included iPhone], so you can see what the camera sees, pretty much. You can also see where you’ve covered on a GPS map.
Greg Corke: Once the data is in the Looq cloud and processed it can then be accessed and viewed through a standard web browser. As we’re talking about huge gigabyte datasets, is data streamed on demand and cached locally?
Dominique Meyer: It is all streaming-based. There is dynamic loading and caching in a web framework that allows fast and easy navigation. It also means that anyone with a computer and a web browser can access that data. And that’s an important part – not needing 128 GB of RAM on your machine.
We want to make it accessible. And that’s why we prioritise that streaming architecture to make sure that anyone and everyone in an organisation can succeed with that tool.
Greg Corke: Are you working to integrate with other platforms, to offer seamless data transfer with something like Autodesk Construction Cloud or any of the Bentley platforms?
Dominique Meyer: Absolutely. We’re working on a number of integrations currently. We’re excited to be building those out and sharing when the time is right.
Greg Corke: Finally, Looq currently has a big focus on the electric utility sector. You’ve talked publicly about oil and gas, industrial automation, civil engineering, and real estate going forward. Is that a case of training the AI on the typical objects within those industries, in order to benefit most from the technology?
Dominique Meyer: In part, yes. There’s the AI component, but there’s also the workflow component, there’s also the business strategy behind it. Today, we’re very excited by helping engineering companies in the electric utility space, succeed at transitioning the grid into a renewable grid, into a sustainable grid. And that’s really where our initial focus is.
A lot of the downstream applications in the CAD, engineering, operations aspects can benefit from Looq, and that’s where, after expanding past the electric utilities, we see great opportunities in different areas.
Oil and gas is a sector where traditionally there has been incredible demand for very high accuracy systems on the laser side, so fitting into there I think is super exciting. But then also going more into civil engineering and real estate that have a lot of structural components that we will be able to apply our AI capabilities to enhance engineering workflows.
A lot of the work in, for example, concrete estimation and structural integrity, right, some of it can be driven through image-based analytics and that’s where Looq really can very likely provide a lot of value in the future. And we’re excited about going in those directions as we grow the company and support the markets.
