Augmented Reality Guided Surge: Next Generation Fluorescent and Image Guided Surgery

by Michael Riemer

Reduce the time and cost while improving patient outcomes by revolutionizing visualizations during surgery using augmented reality
Arlington, VA United States Medical Device Connected Health

All Team Company Physicians Hospital Innovation Details Supporters Comments Updates

About our project

The problem we solve: Other than infection, tissue differentiation is the biggest surgical challenge – where, what and how much to cut. Verbal and visual distractions from 2D monitors lengthen procedures and negatively impact patient outcomes. Current fluorescent guided surgery (FGS) approaches have several major shortcomings: 1) cost; 2) usability; 3) complexity; and 4) low adoption rates. Current FGS solutions are expensive ($1.3M to $2.3M more expensive over 3 years based on 20 surgeries per week and 40 surgical weeks). Current FGS solutions have several usability concerns: black and white, 2D images outside of the surgical field, single focal point, requirement to turn off lights in order to obtain fluorescence, bulky hardy requires extra time (15-30 minutes per procedure) to set up and mechanical arm for camera and light sources are hard to maneuver. The use of external monitors.

About our solution: With AR Imaging, a surgeon puts on AR-enabled glasses (digital loupes) with variable magnification (2x-8x), adjustable color contrasts and voice navigation. Vivid 3D color images of the surgical field and fluorescent imaging significantly improve tissue differentiation and depth perception. Preoperative diagnostic images (e.g. CT, MRI) superimposed on the patient improve situational awareness of the surrounding structures and organs. By revolutionizing clinician visualizations and situational awareness, AR Imaging is focused on eliminating verbal and visual distractions and improving decision making to reduces the time and cost while improving patient outcomes from medical and surgical procedures.

Progress to date:
  • ~$1.3M in grants (non-dilutive capital) has been received over the last three years
  • Working prototype
  • Completed animal studies (where key algorithms were tested and verified)
  • 1 IRB protocol approved (sentinel lymph node mapping) and surgeries being performed at the Cleveland Clinic (CCF)
    • Approved IRBs at CCF go through a lengthy process that also includes Case Western Reserve Medical School, University Hospitals and Metro General
    • Initial surgery performed by Dr. Stephen R. Grobmyer, Director, Surgical Oncology at the Cleveland Clinic.
  • 1 IRB protocol approved for clinical trial (Flap perfusion viability in plastic surgery) but not yet started at CCF
  • 1 IRB protocol in final phase of approval (Basal Cell Carcinoma) process at CCF.
  • 3 PCT patent applications, the first of which has had office action and several key claims already approved (AR Imaging has the exclusive WW license from Univ. of Akron)
  • "Low" risk FDA approval path (class II device) based on strong existing predicates
  • Experienced management team including business executives with many successful exits, AR/VR manufacturing expertise as well as leading medical and scientific expertise
  • Low" risk market entry due to payment and reimbursement (DRG and ICD) codes already in place from prior predicate devices as well as validate peer reviewed papers on value of fluorescent guided surgery
  •  

 

About Our Team

Creator: Michael Riemer

Location: Virginia

Bio: I am a technology business executive and "accidental" serial entrepreneur who left medical school to start (and exit) my first company. 32 years of B2B enterprise software leadership experience in product management, marketing and business development including 3 exits. Last 10 years leading Industrial IoT and field service applications, platforms and digital ecosystems for connected vehicles and heavy equipment Early executive at Nextel (1993). 7+ years running products (handset, services, data, applications) including creating/launching GTM segment strategy (7 segments, 25M Pops) that delivered $70 ARPU (industry highest), 2.5% churn (industry lowest), 4.5M subscribers (record growth) and 3500% increase in market cap. Unique combination of software, hardware and telecom product management with more than 20 product launches and 5 granted patents Broad industry experience (financial services, heavy equipment, transportation, cybersecurity, telecom, field service). Extensive technology background (IoT, hardware, software, cloud, mobile, voice recognition, AI/ML, Predictive & Prescriptive Analytics) generating >$200M in software contract value.

Title: Chief Executive Officer

About Team Members

Frank Papay
Chief Medical Officer, MD, MS BME, FACS, FAAP
Biography: Chairman, Dermatology and Plastic Surgery Institute at the Cleveland Clinic. Performed first near-total face transplant in US and commercialized first neuromodulation implant device.
Title: Chief Medical Officer
Advanced Degree(s): MD, MS BME, FACS, FAAP
Twitter: @4arimaging
LinkedIn: https://www.linkedin.com/in/frank-a-papay-md-3193187/

Yang Liu
CTO, PhD Biomedical Engineering
Biography: Assistant Professor, Biomedical Engineering, University of Akron Expert in medical virtual reality and intraoperative imaging; Principal investigator of multiple research grants from US Air Force, Army, NASA, ODOD, etc; Young Investigator award recipient from International Society of Computer Aided Surgery.
Title: CTO
Advanced Degree(s): PhD Biomedical Engineering
Twitter: @4arimaging
LinkedIn: https://www.linkedin.com/in/-yangliu/

Andrew Moan
CPO, MBA, Finance
Biography: General Manager, Cleveland Clinic – Digital Innovations (AR / VR / AI) Expert in product development for AR / VR / AI. 15+ years in hi-tech focusing on simulation, computer hardware development, software design and business evaluation. Previous employment at Microsoft, Apple, and McKinsey & Co.
Title: CPO
Advanced Degree(s): MBA, Finance
Twitter: @4arimaging
LinkedIn: https://www.linkedin.com/in/andrewmoan/

About Our Company

AR Imaging

Location: 4650 Washington Blvd
Apt 118
Arlington, Virginia 22201

Website: https://www.arimaging.com

Blog: https://www.arimaging.com/blogs/

Twitter: @4arimaging

Facebook: https://www.facebook.com/4arimaging/

Product Stage: Prototype/MVP

YTD Sales: Working on it

Employees: 3-5

How We Help Physicians

AR Imaging uses augmented reality to revolutionize clinical visualizations to enhance tissue differentiation.  We do this by providing AR-enabled glasses (digital loupes), that enable the surgeon to view 3D color images of the patient with coregistration of fluorescent contrast agents and pre-operative diagnostic images (e.g. CT, MRI) superimposed on the surgical field.

Fluorescent imaging helps quickly identify tissue perfusion (viability) and helps identify cancerous areas (they light up like a Christmas tree). The superimposed preoperative images provide a highly effective image guided surgery solution that improves decision making and situational awareness.

All targeted procedures have current solutions and related DRGs and ICDs for reimbursements and payments associated with fluorescent and image guided surgery. In addition to providing a major cost savings, these solutions have other shortcomings that impact surgical experiences.  This includes:

  • Procedures require clinicians to view multiple 2D monitors, creating eye-hand coordination and depth perception issues that can increase procedure time & reduce quality of outcomes.
  • The limited ability to differentiate tissue and accurately gauge perfusion can negatively impact decision-making, leading to increased complications, patient morbidity and reoperation rates.
  • Current attempts to improve tissue differentiation and perfusion are challenged by workflow issues from cumbersome equipment.
  • Use of radioisotopes for sentinel lymph node mapping have significant purchase and disposal costs as well as exposing patients and staff to well-known health risk.
  • Preoperative images, patient vitals and other data that are critical to improved decision-making still require visual and verbal distractions during procedures.
  • Current loupes have a single focal point and have shown to cause chronic pain and related issues for surgeons.

Specifically, the following value for each user that addresses their respective pain point(s) is listed in the table below:

User or Patient

Pain

Value

Hospital

  • Costly equipment
  • OR capacity
  • Disposable costs
  • $1.3M to $2.4M cost savings over 3 years
  • Est. 10-50% reduction in procedure time (reduced anesthesiologist costs, increased capacity, etc.)
  • Reduced complication costs ($10K to $100s K per event)

Surgeon

  • Neck discomfort
  • Workflow impacts and usability issues (e.g. lights off)
  • Procedure capacity
  • Verbal and visual distractions
  • Poor tissue differentiation
  • Radiation exposure
  • Multiple 2D monitors
  • Verbal and visual distractions
  • Auto-focus and variable magnification lessen neck issues
  • Reduced stress and improved decision-making increase quality and reduces time – driving up patient satisfaction
  • Eliminates radiation exposure
  • Improves communications and eliminates distractions

Patient

  • Morbidity due to excessive tissue removal, surgical errors, higher reoperation rates and radiation exposure
  • Shorter surgeries, less blood loss, less morbidity, fewer reoperations, etc.
  • Eliminates radiation exposure

Others in OR

  • Multiple 2D monitors
  • Verbal and visual distractions
  • Improves communications and eliminates distractions

How We Help Hospitals

Hospital

  • Costly equipment
  • OR capacity
  • Disposable costs
  • $1.3M to $2.4M cost savings over 3 years
  • Est. 10-50% reduction in procedure time (reduced anesthesiologist costs, increased capacity, etc.)
  • Reduced complication costs ($10K to $100s K per event)

Innovation Details

Intellectual Property Summary

3 PCT patent applications, the first of which has had office action and several key claims already approved. AR Imaging has the exclusive WW license from Univ. of Akron.

PCT/US2014/062454 / WO2015061793A1

Multipurpose imaging and display system

A multi-purpose imaging and display system includes a display; a detector coupled to the display and having a field of view; and a filter communicating with the detector. The field of view is imaged by the detector through the filter, the filter configured to be sensitive to a first frequency spectrum, so the detector displays only objects within the field of view on the detector that emit one or more frequencies within the first frequency spectrum. The detector and filter can work together in different operational states or modes for acquiring image data of a target object under investigation. A computing device can be included to process acquired image data, and communication interfaces can be employed to achieve networking of multiple systems. A peripheral interface allows a plurality of peripheral devices to be selectively added to tailor the data acquisition and display capabilities of the imaging and display system.

PCT/US2014/062468 /  WO2015126466 A1

Imaging and display system for guiding medical interventions

An imaging and display system for guiding medical interventions includes a wearable display, such as a goggle display, for viewing by a user. The display presents a composite, or combined image that includes pre-operative surgical navigation images, intraoperative images, and in-vivo microscopy images or sensing data. The pre-operative images are acquired from scanners, such as MRI and CT scanners, while the intra-operative images are acquired in real-time from a camera system carried by the goggle display for imaging the patient being treated so as to acquire intraoperative images, such as fluorescence images. A probe, such as a microscopy probe, is used to acquire in-vivo data. Additionally, the intra-operative and in-vivo images are acquired using tracking and registration techniques to align them with the pre-operative image and the patient to form a composite image for display by the goggle display.

PCT/US2016/016991 / WO2016127173A1

Optical imaging system and methods thereof

An optical imaging system utilizes a three-dimensional (3D) light scanner to capture topography information, color reflectance information, and fluorescence information of a target object being imaged, such as a surgical patient. The system also utilizes the topography information of the target object to perform an image mapping process to project the captured fluorescence or other intraoperative images back onto the target object with enhanced definition or sharpness. Additionally, the system utilizes the topography information of the target object to co-register two or more images, such as a color image of the target object with a fluorescence image for presentation on a display or for projection back onto the target object.

Clinical Information

Animal studies completed and published.

A miniature wearable optical imaging system for guiding surgeries

Image guidance can result in improved surgical outcomes, shorter operating times as well as a reduced likelihood of requiring a follow-up surgery for various medical interventions. Many intraoperative imaging systems utilize 2D computer monitors, making it difficult to correlate the surgical landscape with the displayed functional information as well as potentially distracting the surgeon. To address this issue, a miniature, wearable Near Infrared (NIR) fluorescent imaging system entitled Stereoscopic Optical Imaging Goggle is developed. The system is made up of two imaging sensors affixed to a wearable stereoscopic display, providing the surgeon with functional data in 3 dimensions with depth perception. We have characterized the system’s optical properties and fluorescent detection limits. In addition, we have demonstrated the efficacy of the system during surgical studies in chicken. We have found that the system can resolve fluorescent structures down to 0.25mm. The system was successfully guided the excision of fluorescent tissue from a chicken. To the best of our knowledge, the Stereoscopic Optical Imaging Goggle is the first wearable wide-field fluorescence imaging system that offers stereoscopic imaging capability and 3D depth perception.

Stereoscopic Integrated Imaging Goggles for Multimodal Intraoperative Image Guidance

We have developed novel stereoscopic wearable multimodal intraoperative imaging and display systems entitled Integrated Imaging Goggles for guiding surgeries. The prototype systems offer real time stereoscopic fluorescence imaging and color reflectance imaging capacity, along with in vivo handheld microscopy and ultrasound imaging. With the Integrated Imaging Goggle, both wide-field fluorescence imaging and in vivo microscopy are provided. The real time ultrasound images can also be presented in the goggle display. Furthermore, real time goggle-to-goggle stereoscopic video sharing is demonstrated, which can greatly facilitate telemedicine. In this paper, the prototype systems are described, characterized and tested in surgeries in biological tissues ex vivo. We have found that the system can detect fluorescent targets with as low as 60 nM indocyanine green and can resolve structures down to 0.25 mm with large FOV stereoscopic imaging. The system has successfully guided simulated cancer surgeries in chicken. The Integrated Imaging Goggle is novel in 4 aspects: it is (a) the first wearable stereoscopic wide-field intraoperative fluorescence imaging and display system, (b) the first wearable system offering both large FOV and microscopic imaging simultaneously, (c) the first wearable system that offers both ultrasound imaging and fluorescence imaging capacities, and (d) the first demonstration of goggle-to-goggle communication to share stereoscopic views for medical guidance.

Fluorescence Imaging Topography Scanning System for intraoperative multimodal imaging

Fluorescence imaging is a powerful technique with diverse applications in intraoperative settings. Visualization of three dimensional (3D) structures and depth assessment of lesions, however, are oftentimes limited in planar fluorescence imaging systems. In this study, a novel Fluorescence Imaging Topography Scanning (FITS) system has been developed, which offers color reflectance imaging, fluorescence imaging and surface topography scanning capabilities. The system is compact and portable, and thus suitable for deployment in the operating room without disturbing the surgical flow. For system performance, parameters including near infrared fluorescence detection limit, contrast transfer functions and topography depth resolution were characterized. The developed system was tested in chicken tissues ex vivo with simulated tumors for intraoperative imaging. We subsequently conducted in vivo multimodal imaging of sentinel lymph nodes in mice using FITS and PET/CT. The PET/CT/optical multimodal images were co-registered and conveniently presented to users to guide surgeries. Our results show that the developed system can facilitate multimodal intraoperative imaging.

Intraoperative Fluorescence Imaging and Multimodal Surgical Navigation Using Goggle System

Intraoperative imaging is an invaluable tool in many surgical procedures. We have developed a wearable stereoscopic imaging and display system entitled Integrated Imaging Goggle, which can provide real-time multimodal image guidance. With the Integrated Imaging Goggle, wide field-of-view fluorescence imaging is tracked and registered with intraoperative ultrasound imaging and preoperative tomography-based surgical navigation, to provide integrated multimodal imaging capabilities in real-time. Herein we describe the system instrumentation and the methods of using the Integrated Imaging Goggle to guide surgeries.

AR Imaging has a working prototype being used in surgery in trials at the Cleveland Clinic. Initial surgery performed by Dr. Stephen R. Grobmyer, Director, Surgical Oncology at the Cleveland Clinic.

The first active IRB is directed toward sentinel node lymph node biopsies for breast cancer (NCT02802553), the second concerns basal/squamous cell carcinoma (skin cancers) auto-fluorescence with Levulon (alpha-levulinic acid) versus actinic keratosis differentiation and the third submitted IRB is perfusion of reconstruction flaps in plastic surgery mainly free flaps in breast reconstruction. Approved IRBs at CCF go through a lengthy process that also includes Case Western Reserve Medical School, University Hospitals and Metro General) 

Regulatory Status

Our device is type 2 and will require a 510K. However there are several strong predicates and we have a pre-submission document almost completed

How we will use the funds raised

$1.5M - MVP for fluorescent guided surgery. Finalize the digital loupe design and manufacturer the initial MVP. It will also be used to complete the current trials and obtain FDA and EU approvals. 

15% Operations (FDA/EU and Clinical Trials)
2% R&D
1% Marketing
17% G&A
55% Product Development & Manufacturing
9% Legal

$2.0M - Version 1 launch and support/success teams in place. Post market studies with KOLs. R&D on new tissue and bone/organ visualizations as well software and accessories. Customer success and support teams.

5% Support & Customer Success
9% Operations
11% R&D
7% Sales
6% Marketing
15% G&A
41% Product Development & Manufacturing
6% Legal

Thank You

(This post can be read more easily here)

Return on Heartbeat: The Importance of Being Mission Driven

Why did you join AR Imaging is a frequent question these days. The answer is both simple and personal. 

After 32 years as a serial entrepreneur, I was looking for something more meaningful.  Helping deliver $100,000,000s of software and technology had added little to society.

AR Imaging is also in my hometown, Cleveland, Ohio. (No Browns jokes please.) And, I want to contribute to its blossoming technology and entrepreneurial ecosystem.

It's also exciting to contribute to a solution that can help deliver faster, better and less expensive cancer surgeries.  This is near and dear to me because my mom died of cancer at 45.

Finally, I wanted a new opportunity to create mission-driven values and culture. One that makes people as critical as the product and customer. An opportunity to create measurable value for employees as well as the world around us. 

I call this Return on Heartbeat.

My Journey

After 32 years of early and growth stage ventures, a medical device venture may seem like a big change. 

Yet, my career has crossed many different technologies , applications and industries.

Growing up on the eastside of Cleveland, I attended Beachwood High School. After, I went "pre-med" to the University of Rochester where I majored in Biology and Philosophy

After graduation, Case Western Reserve University, School of Medicine extended an offer. Shortly after accepting, Peter Tippett and I started a software company, Certus International. It was one of the first anti-virus products for PCs.  The company was eventually sold to Symantec. 

Since then, Peter and I have both enjoyed great careers but never found another opportunity to work together, until now.

The Company

In early February 2018, Peter suggested that I look into a company which was then called I-Imaging.  A few weeks later we traveled to Cleveland.  I met the co-founders Dr. Frank Papay and Yang Liu, PhD.  A month later, I was onboard and very excited.

AR Imaging combines many great technologies.  This includes augmented reality, digital health and industrial IoT. It also uses techniques in computer vision as well as image recognition and processing.

But technologies don't solve problems.  

So let's not focus on them for now.

Except to credit AR Imaging's tremendous technical achievements.

Tissue Differentiation Is The Focus

Instead, let's focus on a critical problem - tissue differentiation. 

Besides infection, knowing what, where and how much to cut is a huge challenge. It is one of the oldest and most basic challenges of surgery. It takes year of training and practice to ensure successful outcomes.

AR Imaging's goal is to improve decision-making by improving clinical visualizations of tissue differentiation.  We also happen to be using a bunch of "cool" technologies.

AR Imaging is the next generation of fluorescent and image guide surgery. Improving decision-making and situational awareness. Our objective is to improve patient outcomes and lessen the cost and time of the procedures.

We are very optimistic of our commercial success but what about return on heartbeat?

World Mission View

By 2050, the US will see an 82% growth of individuals over the age 65. China will experience 183% in that same timeframe. Europe’s aging projections are similar. Our aging population is increasing the frequency of related procedures including cancer. 

At the same time, the WHO estimates a shortage of 4.3 million physicians, nurses and other health workers. In the US, the AMA estimates a shortage of 91,500 physicians by 2020 and up to 130,600 by the year 2025. 

Contributing to 5 billion people today without access to safe surgery.

AR Imaging is not a panacea to this enormous problem.  

But AR Imaging can help delivery safe surgery solutions around the globe.  

Our small form factor and cost effective approach provides a great delivery platform.  First for fluorescent and image guided surgery. In the future for telehealth, telesurgery and other collaborative solutions.

The Circle of Life

My journey has led me back to my hometown. To the medical industry.  To a mission driven opportunity. To a great return on heartbeat. 

Supporters

Comments

Login to post your comment!
Click here to Login

Updates

    No updates found .

$ 0

pledged of $ 1,500,000 goal

  • 75 Days left

0

Interest
Score

0

Adoption
Score

Michael Riemer
Chief Executive Officer

Proflle
Contact Me


Rewards