May 24, 2019 | Kanwal Raghav is on a mission to design clinical trials that can assess patient outcome based on the various cell-free DNA assays available.
We can now check blood for 450 mutations, Raghav points out, "but does checking 450 mutations in colon cancer improve patient outcome over checking the four or five mutations that we can successfully target? That is a question that we haven't answered in clinical trials."
Raghav is an assistant professor of gastrointestinal medical oncology at the University of Texas MD Anderson Cancer Center. His research efforts are directed toward understanding tumor biology, therapeutic resistance, and cancer biomarkers and development of novel therapeutics for colorectal cancer and unusual tumors of the gastrointestinal tract. He believes randomized control trials are needed to reveal our own biases in patient selection and determine which tests truly improve patient outcome.
On behalf of Diagnostics World News, Christina Lingham spoke with Raghav about the validity of variant testing, how we should improve the reliability of these tests, and what’s at stake.
Editor's note: Christina Lingham, a conference producer at Cambridge Healthtech Institute, is planning a track dedicated to Clinical Applications of Circulating Biomarkers at the upcoming Next Generation Dx Summit in Washington, DC, August 20-22. Raghav will be speaking on the program. Their conversation has been edited for length and clarity.
Diagnostics World News: In the news recently there have been reports of faulty tests with variant calling and discordant data. What do you think is the impact of this finding?
Kanwal Raghav: First, a disclaimer: I'm not a pathologist and I'm not involved in assay development as much as I'm involved in the clinical application of those assays. But the more sensitive you make the assays, there is always a chance that you're going to pick up something that is an artifact. Just to give you an example, when we do normal tissue sequencing and we did it with our older assays, we found KRAS mutations, basically. For example, when a colon cancer patient has KRAS mutations, we know in the sum total of that tumor, KRAS mutations are present. However, when you start doing blood assays, you find even lower levels clones that are shedding DNA into the blood, so you catch more than what is seen in the tissue.
Now some amount of it is related to assay sensitivity; some amount of it is related to tissue pathogenesis. Right? Unfortunately, I don't think we have a lot of evidence—what can be considered good quality evidence—that would be able to feel those parts out. The only way to do that would be to biopsy patients in multiple areas and then get blood, and that is just not clinically feasible. From a clinical perspective, my feeling is that at the end of the day we have to do studies and clinical trials that can show how we can translate these assays into functional outcomes in patients. And I think that is what is lacking in the field.
You have assays that show you, "Here is the tissue that shows X gene. Here is the blood that shows X gene and Y gene." And then a discrepancy of 5% between tissue and blood. But does that discrepancy really translate into better management of patients or worse management of patients? That's where the field should be moving.
How do you think we can go about improving the reliability of tests?
As far as the reliability of the test is concerned, I think the buzzword is validation studies. This can be done in two ways. You can either do dedicated trials for these validations, prospectively with good standard operating procedures, and really large, well-annotated cohort of patients, or you can do retrospective studies.
For example, one way of doing the assays for me is I have a cohort of patients, 500 patients, let's say. But I have tissue tests, right? And now I want to validate the ctDNA assays, so I'll do a ctDNA assay of the blood that was collected by the patients a long time ago. That's the fastest way of doing something because doing it in real time takes time and effort and money. Ideally, I’ll take every colon cancer patient that walks in clinic. I'm going to do a biopsy. I'm going to collect blood. I'm going to make sure that that blood is collected according to the standard operating procedures for the assay. And then I'll do this over a period of two years to get a sufficient sample size—500 patients or 600 patients or 1,000 patients.
Unfortunately, most of the time, that's not how you research these techniques. Most of the time, blood samples are collected before tissue samples. The technique used on those tissue samples is different from the technique that is for the blood assay. So that's why we find all these discrepancies.
Moreover, there are problems of patient selection. For example, if I took 100 colon cancer patients that come through my clinic, they will have disparate and varying level of disease. We know from studies that the sensitivity of the assay or the ability of the ctDNA assay to find the mutation in the blood can go down if you have very low volume disease. If your disease is small amounts of disease that is found in the body, you may not find as many mutations. If I have already collected blood on 400 patients before I'm expecting to do ctDNA analysis, by default, do you know what I would do? If there was somebody with very low volume disease I would say, "Oh, let's not collect blood on this patient because you know, what’s the chance that it would be positive." There is the issue of patient selection.
These have been the challenges in validation. I think the best way to increase the reliability of these assays is to improve the validation of these assays using prospective clinical trials, and not self-selecting our patients. When you select your patients, then they don't represent what your actual patients look like.
To give you a classic example – There is a mutation in colon cancer called BRAF mutation. The survival of patients with BRAF mutation in the community is very poor. But in our own groups—MD Anderson or Memorial Sloane or big research centers that are actually involved in doing this research with ctDNA assays—we already have a selection bias in a patient that comes to us. If somebody was doing really poorly, they would not make it through large tertiary care centers. Our survival looks much better, but it’s not because of treatment, it can also be because of selection of patients.
The problem is that the research for these assays is being done in select centers. But where it is being applied? It's being applied in the community.
Right. In your opinion, who are the gatekeepers ensuring test quality, and what are they doing to improve and ensure future efficacy?
I think this is more of a pathology question because pathologists are involved in the assay development. Recently, of course, the FDA has made efforts to get companion diagnostics so when you're doing the study and you want to establish an assay as a standard, you need to do it presently within the clinical trial that you're applying for FDA approval. It's been shared between pathologist, assay development teams, as well as the FDA, but most of this validation and reliability at this point is still with the companies that are developing the assays.
How will it impact consumers that we don't have an established mechanism for ensuring test quality, that it falls between those three groups: the pathologists, and the development people, and FDA.
We have a lack of large, well-designed studies that can actually assess the impact of doing these tests on patient outcomes, and that is central and key to a judicious utilization of these tests. There are three questions that arise. Number one: Can we do the test? You don't need a trial for that. Can you collect blood from every patient? Sure. You can absolutely do that. That's the strength of the assay. If you can test something in the blood, you know that there is a feasibility. If I said, "Oh. I need to devise a test which requires three biopsies of the liver." You will have to do a feasibility study because you want to know whether you can actually do that in patients, but collecting blood should not be a problem because it has been done in the past. That's the first question.
The second question that comes up with any assay in ctDNA is: Does is actually test what it is supposed to test? Can it find BRAF mutations when it is designed to find BRAF mutations. And the answer is yes. It can do that because there are enough studies to show that if you have BRAF mutation in the blood you will be able to capture it. In fact, if anything we are dealing with oversensitivity issues where we are finding stuff that may not be relevant to patient care in general. So those two questions have been answered.
The third question is a clinical question: Does this test really change patient outcome? And that has not been established yet because when you go into tissue next-generation sequencing we say, "Ok, we should do NGS on all our patients who have cancer to test for as many mutations as possible." For example, our Foundation Medicine test checks more than 450 mutations. There are multiple platforms for this. We can check 450 mutations, but does checking 450 mutations in colon cancer improve patient outcome over checking the four or five mutations that we can successfully target? That is a question that we haven't answered in clinical trials.
And the same thing goes for ctDNA. You can capture more mutations in ctDNA, and that can tell you that the tumor is more heterogeneous. That's not hard to find. The question is: Have we done trials to show that seeing more of those mutations, either adversely or in a better way affect patient outcomes? One-off cases do not count as evidence; one case does not data make.
How are you working to address these challenges?
Our efforts are mostly around designing clinical trials that can assess patient outcome based on the assays. We don't directly test reliability of these assays, but in an indirect way, if you are able to associate these assays with patient outcome in prospective trials, then that is a test of reliability because you're doing it in real time.
For example, in today's world, the standard treatment for cancer is you start a treatment and then two months later or three months later, you do a CT scan, and then you see whether the patient is responding, meaning the tumor is decreasing in size, or is the tumor increasing in size. It's just not feasible to do a CT scan every two weeks or three weeks. It's very costly, you're exposing patients to radiation, and it's not a very sensitive test to detect any difference.
But we have shown in our data that if you check ctDNA of a patient when you start a treatment and then you check the ctDNA within 15 days of starting that treatment, if the ctDNA goes down, then you are more likely to respond, and if the ctDNA goes up, you are more likely to progress.
We are trying to design randomized trials. Half the patients get what is standard: You get the treatment and you get a scan in three months, and then you measure how well that entire group does. And in the other group, you take ctDNA, and adjust treatment based on response. And then at the end of that experiment, you measure how these two groups did. This is the only way of ensuring that catching that progression two and a half months earlier really changes the outcome of the patient.
There can be three outcomes in this. Number one: It doesn't change the outcome, and patients survive exactly the same amount of time. In this case, even though the assay can perform, it doesn't really change the outcomes. The second option is it actually does improve outcomes because patients get less-toxic treatments, and the ones that get treatment are the ones that are more likely to benefit. But there's a third possibility, and the third possibility is that it does not improve the outcome, in fact, it makes it worse because now you've taken patients off and maybe there were 10% of patients that were likely to benefit, and even they are not benefiting.
There's no way any test can be 100% perfect. I think that these kinds of trials will define how we can actually clinically apply this test. Right now, the field is operating with minimal evidence and more extrapolation that if you find something early, you should be able to improve outcomes. But that has not always been true in cancer.
We need to establish good, randomized prospective evidence associating ctDNA assays with patient outcomes directly. Then we should move forward, and that's what we are trying.