Challenge – Analysing Contents of Photos

Sky Discovery blog authored by Jeff Jarrett and Anderson Hind

A few years ago we worked on a construction litigation that revolved around a failure in a hydroelectric dam.  Specifically it came down to cracks in the dam wall. The legal team was busy organising experts, and they needed as much evidence they could find relating to these defects in order to brief them.  In this instance, they knew the company has captured the evidence through visual photos. The challenge however was trying to find the photos that showed the defect. Like many construction litigation matters, there was a huge number of documents, and within the set a large number of photographs.Typically we can use search terms on emails, word documents etc, but for photos the eDiscovery world typically comes up a bit short.  If we’re lucky we can get a created or last saved date out of the metadata, and if we’re really lucky maybe even get GPS coordinates. When it comes to photos and eDiscovery that’s about where it ends. If you want a clue to what’s going on in an image, unfortunately you just have to review them manually. For this matter a few years ago, that was the solution we went with. Needless to say it was time-consuming and expensive in terms of eyeballs.

So we’ve known about this photo problem in litigation for some time without being able to offer much in the way of help. However, recently there has been an interesting development in the cloud provider space. All of the big providers, Amazon Web Services, Microsoft Azure and Google Cloud have been developing new services and offering them to the public via their clouds and accessible by developers. One that piqued our interest was object recognition. You can submit a photo and it will compare that photo to its database of millions of images. If anything looks similar, it gives you a matching label with a confidence score.

The service is pretty helpful across a variety of images:

The image recognition returns this as Building, Bridge, Suspension Bridge, Town, Urban, Metropolis & City.

This comes back as Slope, Nature, Mountain Range, Outdoors, Mountain, Peak, Plant, Fir, Tree & Countryside.

 

This returns Human, Person, Clothing, Apparel, Outdoors, Nature, Art, Field, Painting & Water.

 

We of course had to know if it would have helped us with the defects in the dam wall matter:

This came back with the term “Dam” at 95% confidence, so we at least knew it could point us in the right direction.

The exciting thing about these services for eDiscovery is it gives us a tool to be able to meaningfully search photos f

or the first time.  We’ve been experimenting with this service for a while ad-hoc, but recently actually wrapped it up into a nice solution called Sky Photo Search. This lets the user effectively click one button inside their eDiscovery plat

form, then behind the scenes Sky Photo Search securely connects with the required cloud provider (AWS, etc), gets the photo label results and puts them back into the eDiscovery platform.  When it’s done any photo in the workspace is then text searchable.

We’ve since expanded the solution to also find other traditionally non-searchable items. Using this same solution we’re now able to search documents for handwriting and then transcribe the results:

We’ve found it’s actually much easier for the reviewer to read the transcribed text than the original human scrawl.  We haven’t actually put through any doctor’s notes yet, those may be just too far beyond saving.

In summary, this solution can offer a significant improvement over manual review of photos for construction & infrastructure matters. If you have any thoughts or comments please do share.

Contact details

For further information on Sky Discovery’s eDiscovery services, please contact your Sky Discovery Consultant on telephone: +61 2 9293 2999 or email: sales@skydiscovery.com.au

About the authors

Jeff Jarrett, Director of Innovation

As the Director of the Innovation team, Jeff spearheads the development and application of custom-built solutions for client, consultant and stakeholder workflow needs.  With Jeff’s experience in eDiscovery, Relativity and client delivery, he is able to identify potential ‘pain points’ in the workflow and develop innovative solutions to streamline efficiency.  Many of these innovative solutions are rolled out to benefit all clients so they save time and costs.

Anderson Hind, Director of Technology

As the Director of Technology, Anderson has significant experience working with Partners, Senior Associates and CIO’s to provide strategic advice and innovative solutions to resolve challenging situations.  He has worked with a variety of software to design solutions and integrate the results into Relativity and the eDiscovery process.

About Sky Discovery

Sky Discovery is a leading independent Australian eDiscovery firm. We focus on building strategic relationships with leading mid-tier, specialist and international law firms in Australia that require outstanding eDiscovery services to achieve successful outcomes on disputes, investigations and regulatory matters. We are renowned in the Australian market for having one of the largest teams of eDiscovery experts, a 24/7 service model and, technological and service innovations that underpin speed and quality whilst providing demonstrable cost savings.

Leave a comment