By Deborah Borfitz
December 14, 2021 | Histopathology is destined to go through the same sort of digital transformation that occurred in the home entertainment industry with the wholesale switchover from in-person visits at Blockbuster stores to anytime movie availability from streaming service Netflix, according to Aydogan Ozcan, professor and chair of engineering innovation in the electrical and computer engineering department at the University of California, Los Angeles (UCLA). Glass slides are going to be replaced by digital pathology and virtual staining, speeding time to answers from roughly a week to hours.
This new virtual histology process “bypasses several standard steps typically used for diagnosis—including skin biopsy, tissue fixation, processing, sectioning and histochemical staining,” says Ozcan. The enabling technologies include reflectance confocal microscopy (RCM), a noninvasive optical technology that produces images of intact skin without having to externally label it with toxic stains, paired with a newly developed virtual histology platform powered by artificial intelligence (AI).
Used together, the technologies can replicate the look of conventionally biopsied, histochemically stained skin sections on microscope slides. The combo could one day reduce the need for skin biopsies and provide rapid diagnosis of malignant skin tumors, Ozcan says.
Previously, the virtual histology platform was applied to microscopy slides that contained unstained tissue acquired through a biopsy. But, as reported in Light: Science & Applications (DOI: 10.1038/s41377-021-00674-8), it was recently applied for the first time to intact, unbiopsied tissue.
"The only tool currently available to most dermatologists for diagnosing skin lesions is a dermatoscope, which magnifies light on the skin to pick up patterns," says Philip Scumpia, M.D., assistant professor of dermatology and dermatopathology at UCLA and the West Los Angeles Veterans Affairs (VA) Hospital and member of the UCLA Jonsson Comprehensive Cancer Center. “As dermatologists we are missing this huge opportunity… where we can incorporate mole mapping and reflectance confocal imaging with virtual histology… to track lesions over time.”
Ozcan is co-founder of Pictor labs, a UCLA spinoff, which is seeking to commercialize virtual histological staining of label-free tissue. The system is not yet being used on human tissue to make primary diagnoses, which would require its approval as a class III medical device by the U.S. Food and Drug Administration (FDA), he says.
Go-To-Market Vision
The FDA has already started discussions around incorporating AI into algorithms for radiology, Scumpia says, and the virtual histology platform using RCM will likely be an “offshoot” of that. “The good news is once we start collecting more samples and these samples get biopsied, we’ll have what the actual pathology is for the lesion, and we can see how the RCM plus virtual histology matches up… and how this would work in clinical practice.”
The FDA has approved numerous digital pathology medical devices that could serve as predicates for similar systems entering the market, but the combination of digital pathology and virtual staining is novel. Interestingly, adds Ozcan, use of the system for second opinion teleconsultations falls outside the agency’s purview and is instead regulated by guidelines issued by the College of American Pathologists.
In Europe and Asia, where doctors commonly seek second opinions via telepathology services, virtual histology could add value to those teleconsults, he imagines. The approach would help “democratize” access to technologies that are otherwise not globally available.
Standardization of tissue staining using AI-powered label-free imaging technologies could also broaden access to high-quality staining, says Ozcan. At present, tissue staining done even by top-notch histopathology labs is surprisingly random, with uneven quality and the chemicals used in the process are toxic.
“During the [AI] training process, we first understood how inconsistent standard pathology is in terms of repeatability,” Ozcan says. About one-quarter of tissue samples processed at state-of-the-art labs in Southern California came in so torn, distorted, or poorly stained that they were unusable for creating ground truth images. The situation would be predictably much worse at less resource-rich labs in, for example, developing countries.
Moreover, as the pandemic has highlighted, the chemicals used for tissue staining are subject to supply chain disruptions, he continues. Most stains are produced in India, whose government last year imposed one of the world’s longest COVID-related lockdowns. Tissue staining aids in visualizing different structures in contrasting colors and, in its absence, patients’ condition can be inaccurately diagnosed.
Imagine instead if doctors from anywhere in the world—including primary care clinics in rural regions of the U.S.—could send an RCM image with minimal to no processing to teleconsulting pathologists in Los Angeles for virtual staining and a second opinion on whether a lesion is cancerous and requires a biopsy, says Scumpia. For patients being treated at various VA hospitals in California, it could save them the inconvenience of a three- or four-hour drive for an unnecessary excision.
Optical Technology
RCM has been around since the early 2000s but is “only now starting to pick up steam in dermatology,” Scumpia says. In the U.S., its use is limited to a handful of dermatology experts who can read grayscale images and mainly to “suggest” a diagnosis. Unlike standard histology, the images they produce lack nuclear features.
Virtual histology that can enhance RCM images to the point where they resemble conventional histology using the two dyes, hematoxylin and eosin (H&E), should impact popularity of the imaging modality among dermatologists and dermatopathologists, he continues. The onus is on researchers to show that the approach can differentiate malignancies from benign conditions such as squamous cell carcinoma, pseudoepitheliomatous hyperplasia, nevi, or acidic keratosis.
Even with traditional histology and pathology, adds Scumpia, “there are some things I see under the microscope that I can’t give you a [definitive] diagnosis on.” And “some of those issues will exist whether you’re using RCM or you’re using biopsied skin.”
Ozcan says he is optimistic that any challenges that arise will be addressable by AI as it consumes, and learns from, data generated by more and more sample types. Mistakes are a “great thing” because they can be tapped to improve performance of the convolutional neural network powering the virtual histology platform.
The newly published, peer-reviewed paper was about 40 pages long (inclusive of supplemental materials) because of the rigorous analyses that have already been done with the deep-learning-based virtual staining algorithm, he notes. “We have tweaked the networks, we have tweaked the [input and output] channels, we have tweaked the training process, the data acquisition process… and more data will make this shine.”
Digital Staining
For at least the past five decades, scientists have tried different forms of microscopes to enable digital staining of tissue samples without acceptable performance, says Ozcan. But most of those earlier efforts stalled out prior to the digital revolution around deep learning, highlighted by the resurgence in neural networks and the development of AI training tools by Google and Facebook and GPUs (courtesy of the computer game industry) well suited to the execution of energy-hungry machine learning algorithms.
Virtual staining of a whole slide image can now be done in about a minute—almost real time, he continues. Across a repeated line of testing with clinicians and pathologists specializing in different tissue types, the results have proven indistinguishable from traditionally stained tissue samples.
Pathologists, when challenged, have similarly been unable to tell when they’re looking at the image of a stained tissue slice on a glass slide or a virtually stained tissue image, says Ozcan. Whether they can or can’t “is like flipping a coin.”
Producing a high-quality image requires that the tissue samples be properly processed, which tends to be a problem at many smaller labs that may not have a good histotechnologist, says Scumpia, who joined Ozcan’s virtual staining efforts three years ago.
Getting to the point of making biopsy-free in vivo virtual histology of the skin a reality has been a longer, six-year journey. Biopsies are by nature “very laborious and tedious,” says Ozcan, and involve staining very thin slices of the sample (on the order of 3 to 4 microns) so different tissue features can be visibly identified by their color and texture. It’s a practice that has been done in medicine for over a century now.
Previously, Ozcan and his UCLA colleagues demonstrated how those biopsy samples could be virtually stained and, in doing so, significantly cut down the time it takes to get images to pathologists for their diagnostic decision. In a succession of high-profile papers, they proved out the approach on a variety of organs and different stain types, including H&E, periodic acid–Schiff (PAS), Masson’s trichrome, and Jones stain (a methenamine silver-PAS stain).
They then tackled the more difficult problem of eliminating the biopsy step using RCM images of the skin. Combining virtual histology and AI, the research team transformed primitive images of the skin into the type of images coming out of histology labs from H&E stained, biopsied samples.
3D Images
The latest study used RCM to create virtually stained “H&E-like” images from three-dimensional (3D) cross-sections of tissue, Ozcan explains, which was particularly challenging. This is partly because the standard biopsy and histochemical staining processes alter tissue samples, and thus histology results, which make it very difficult to create ground truth images.
Additional complexities were dealing with a rigorous 3D image registration process for the AI training phase and introducing neural networks that work with contrast-enhanced RCM images to enhance the otherwise grayscale, hard-to-visualize information RCM produces, he adds.
It should be a well-rewarded effort. Virtual staining, Ozcan predicts, will eliminate some of the quality control issues that have plagued histopathology. Standardizing tissue staining quality using label-free imaging technologies that are powered by deep learning would also address supply chain concerns and provide a “better, faster, and cheaper” route to time-urgent diagnoses without any unnecessary biopsy or tissue processing.
From Ozcan’s perspective, one important takeaway from the latest study is that the RCM imaging modality is already being used in some dermatology practices—only now, the research team has transformed the 3D grayscale image stacks into more easily interpretable, H&E-like images using two sets of neural networks. Now that bridge has been built, he expects clinical adoption of RCM in dermatology and dermatopathology will accelerate.
RCM is generally used today to create a stack of four or five black-and-white 2D images, says Scumpia. The new virtual histology tool produces virtually stained volumetric image data and can be further advanced in the future to also flip the stained images to the vertical plane that dermatologists and pathologists are used to seeing.
A major future possible use of virtual, biopsy-free histology is for making diagnoses, Scumpia says. As just demonstrated by the UCLA research team, virtual staining of RCM-generated images can suggest pathology in the form of basal cell carcinoma—one of the most diagnosed cancers in the world—as distinct from noncancerous melanocytes within nevi based on its darker pigment coloration.
“When we began looking at the benign nevi, we didn’t know what to expect, because the melanocytes have a completely different reflective pattern than keratocytes [cells that make up the epidermis] and fibroblasts [cells that make up the dermis],” says Scumpia. Intriguingly, AI-based virtual staining was able to reveal images that can be used to differentiate cancerous from noncancerous melanocytes based on those differences without any training whatsoever.
Next Steps
The goal now is to incorporate this virtual histology technology with RCM imaging on other skin neoplasms, he adds. Through a new grant from the VA hospital system, the research team will next look at squamous cell carcinoma and actinic keratosis that often mimic one another.
Ozcan believes it is possible to computationally deal with some of the challenges associated with blood vessels, which can be seen flowing when RCM is used on intact skin. That’s because their structure is so unique and elongated and appear across all the tissue sections being examined.
It could be achieved employing another layer of AI as a “blood vessel extractor” to isolate those regions and extract them out during the virtual staining process, he explains. In fact, the blood vessels could perhaps be of benefit as alignment markers to facilitate better image registration and training of virtual staining networks.
The long-term goal is to provide virtual histology technology that can be built into any device, potentially in combination with other optical imaging systems, says Ozcan. Once the neural network has trained on many more tissue samples, it could run on a computer or network to rapidly transform standard, label-free images into virtual ones.
Future studies will determine if the digital, biopsy-free approach can interface with whole-body imaging and electronic medical records to usher in a new age of “digital dermatology” and change how dermatology is practiced, he adds. The research team will also investigate if pairing their artificial intelligence platform with other AI technologies would further aid in clinical diagnosis.
Excitement is already quite high about the potential utilization of virtual histology in drug development, including toxicology reports, Ozcan says. All the work underway generally applies to animal models as well as humans.
Tissues taken from various animal organs are quite small and many preclinical tests are required by regulators, he notes. Currently, histology-related analyses when done routinely can require millions of slides that each get discarded after staining. By preserving tissue for additional molecular analysis, virtual histology is “like one stone, many birds,” resulting in newfound efficiencies for pharmaceutical companies.