DARPA DSO :: Prophecy Teaming Site

Prophecy
Proposal Abstract Feedback
The Prophecy Broad Agency Announcement (DARPA-BAA 10-40) is soliciting research proposals to develop technologies that predict natural viral evolution. Proposed research should investigate approaches that enable advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice. Abstracts submitted to this solicitation have been reviewed by DARPA against the published criteria. General feedback and common concerns about the work proposed are grouped by category and summarized below. Proposers choosing to submit a full proposal are particularly encouraged to consider these comments carefully during their proposal preparation. Full proposals are due by 4:00 PM (Eastern) on Tuesday, January 4, 2011.

Algorithm(s)

Full proposals must describe the development of mathematical models or algorithms to predict the future state of a viral population, are related to a phenotypic endpoint (e.g., antiviral resistance, immune escape, zoonotic jump), and are experimentally validated by a proxy in vitro system that exceeds the current state of the art. Quantitative assessment of the model using standard statistical assessments, e.g. ROC curves (or similar), is critical to measure algorithm improvement. Proposers are encouraged to use both positive and negative experimental results to refine their algorithm, as possible. Approaches that focus exclusively on parameterization of pre-existing models are unresponsive to the BAA. Proposers are strongly encouraged to read the BAA in detail.
The development of predictive computational algorithms and models of viral evolution informed and driven by data generated from proxy in vitro systems, or data initially derived from archived repositories, is required. Development of computational models and in vitro platforms must be fully integrated with one another. For example, approaches using bioinformatic and structural modeling should be supportive of one another and not independent parallel efforts.
Full proposals should describe how the parameter space of the predictive model matches with the experimental proxy system they plan to use. For example, proposers referring to machine learning approaches should describe: a) how the dimensionality of the parameter space will be accounted for, how this relates to the number of experiments required to obtain an accurate result, b) how many training examples would be needed, c) any strategies to deal with a high dimensionality or linear and non linear dependencies between parameters.
Early validation of the algorithm development is encouraged in the first half of Phase 1. This will be used to support an evidence-based progression to Phase 2 and 3 program goals. This may be accomplished using existing or archival data sets or newly generated experimental data (or both).
Full proposals must describe in detail the specific approaches that will be used for model and/or algorithm development, and include rational for their selection. Proposals that do not specify their approach (e.g., Bayesian inference, Hidden Markov Models, as referenced in the BAA), will be considered non-responsive. It is insufficient to simply list the application of a generic field such as "machine learning".
Full proposals must provide strategies for addressing errors in viral evolution platforms, data analysis, and predictive algorithms. If using a segmented virus model, algorithms should account for the possibility of reassortment or describe expected shortcomings.
Full proposals must estimate the time and resources required for computational approaches. Detailed descriptions of anticipated steps and process burdens as they pertain to accomplishing the stated goals of the proposal are encouraged.

Experimental Platform(s)

Full proposals must describe the appropriateness of the viral model chosen for the goals of this project. Specific positive attributes include, but are not limited to: access to extensive archives of historical viral sequences; simple viral genomes amenable to high throughput sequencing and rare event detection; adequate viral genomic mutation rates such that viral evolution is expected in the time frame of the experiment; and well characterized and representative animal models. Experimental platforms with real time rare genetic event sampling of the viral quasi-species and description of the evolutionary trajectory are encouraged. Moreover, platforms that are flexible and allow scalability to match the viral system under study are encouraged. For instance, platforms should allow flexibility in throughput to study both slow evolving non-segmented viruses as well as rapidly evolving segmented viruses, which may require more sampling and data analysis.
This program anticipates the development of trans-disciplinary approaches to the prediction of viral evolution. Viral evolution platforms must represent a significant advancement in the state of the art. This program is not intended to advance the state of development for existing biosensors or generic biotechnology tools that are not clearly relevant to the development of a predictive algorithm.
Viral diversity in a starting viral population impacts the evolutionary trajectory of that population. Full proposals should include a description of measures to control and/or carefully characterize the viral diversity in starting populations.
Failure to describe how the team's sequencing capability will address quasi-species characterization, rare event detection, or the massive throughput required to characterize the data from the biological validation system is considered non-responsive to the Prophecy BAA. Proposers are encouraged to indicate the expected number of samples and depth of coverage required for accurate description of the quasi-species for the particular virus under study and describe the rationale for arriving at these numbers. In addition, proposers should carefully consider the relevance and appropriateness of their proposed sequencing technologies and should describe strategies to address systemic errors that are specific to a particular sequencing platform (e.g. high error rates in homopolymer regions, which may be addressed by combining data from a second platform) as well as systematic errors associated with sample preparation protocols (e.g. high error rate associated with cDNA synthesis using reverse transcriptase). Moreover, contingency planning is advised for proposers utilizing immature platforms, as delays may arise from unforeseen or unanticipated issues surrounding these as-yet-unproven technologies.
Full proposals must estimate and describe the time and resource requirements for the proposed experimental approaches. Proposers are encouraged to describe the steps and quantify the burden of their process as it pertains to accomplishing the stated goals of the proposal. Please describe the experimental work flow in detail including the timing of sampling, the number of samples to be taken at each time point necessary to detect rare events, sample preparation, and assay(s) to be performed on each sample. Details should include an estimation of the time and resources necessary to perform all assays.
The ability to detect rare mutants in an animal model will likely be limited by the time and resources required to complete individual experiments. This may not provide sufficient throughput necessary for algorithm development. DARPA strongly advises performers to account for this concern. Teams who propose only animal models as an evolutionary platform must clearly defend their ability to achieve the rare event detection and high-throughput goals of the program.
Synthetic viral systems that do not reproduce the dynamic and diverse nature of a viral population will be considered non-responsive if they are not also supported with a relevant dynamic system to validate the approach. Additionally, approaches that do not describe probable evolutionary trajectories will be considered non-responsive.
Full proposals must choose selective pressures that have precedent in modulating or constraining natural viral evolution and/or supply supplementary data that demonstrates the ability of novel selective pressures to imbue the viral evolution platform with evolutionary trajectories that are representative of real world phenomena (e.g., immune, antiviral, host cell type). Therapeutic interventions as a selective pressure should be drawn from and supported by modalities with precedence in the available literature. Effort should be made to develop an iterated cycle between data gathering and algorithm development. As such, the throughput of the phenotype analysis should scale with the rate of the evolutionary platform data acquisition. Time needed to collect sample and perform assays should be minimized to allow for maximized number of iterative cycles in the period of performance.
Any proposals that seek to demonstrate viral evolution platforms that have been intentionally designed to increase pathogenicity (virulence) or otherwise made more dangerous will be viewed as in violation of the Safe Laboratory Practices and potentially the Biological Weapons Convention (http://www.state.gov/www/global/arms/treaties/bwc1.html) and will not be reviewed. Full proposals must be aware of the necessary protocols and comply with the appropriate laboraotry safety requirements (e.g BL1-4). The use of low-threat, vaccine-preventable viruses is preferred; use of high risk viruses incurs considerable cost, challenge and infrastructure and must be justified with detailed rationale.
The choice of in vitro model used should be well substantiated. Disclosure of the decision process will support a successful proposal. For example, if transformed cell culture lines are used, their validity as a natural viral host should be supported by the literature. For primary cell culture, establishing routine controls to avoid contamination by other cells, pathogens, etcetera is critical. in vitro viral evolution platform(s) should be validated using relevant animal and/or clinical studies when available.
A strong rationale supported by available literature for focusing on the evolution of a single viral protein and/or gene (as opposed to whole genome) is strongly recommended.
in vitro random mutagenesis of gene sequences may not reflect the natural diversity of genetic sequences that are generated through in vivo evolution, thereby resulting in the biasing of certain genetic sequences. Proposers must provide appropriate scientific rationale when proposing to generate a priori evolved viral genes or genomes through in vitro random mutagenesis approaches and describe how such approaches recapitulate the natural genetic diversity derived by in vivo evolutionary events. Proposers using in vitro mutagenesis approached are encouraged to also consider using relevant natural evolution systems in parallel to validate results.

Staffing/Expertise

Full proposals must demonstrate staffing and expertise that is relevant to ALL goals of the program with a teaming strategy established prior to submission of the proposal. Successful teams will be composed of experts in each of the experimental/computational approaches proposed. Multiple proposals lack sufficient expertise in the viral systems, algorithm development, sequencing technologies and experimental platforms proposed. The contribution of each team member should be described.
Proposals that rely on award of instrumentation (i.e. sequencing platforms) to achieve program goals are encouraged to instead team with individuals and labs that can offer this expertise and eliminate the significant lag time to instrument arrival, set-up, routine operation, and data management. DARPA is concerned that several proposed sequencing systems are not as yet available and once purchased will incur significant time to refine and ensure quality and reliability

Transition Plan

Full proposals must address a transition plan that will enable the application of their results to the real world for the purposes of predicting viral evolution. As such, full proposals must choose a viral model and evolutionary platform that is appropriate to meet the DARPA mission with eventual applicability to real world surveillance data. For example, platforms that require extensive a priori knowledge of the virus in order to mutate specific viral proteins in isolation or platforms that are not amenable to the study of other infectious diseases (e.g."fossilized" viruses in organism genome) are not appropriate for this program.
Proposers are encouraged to develop approaches that can be adapted to apply to other systems and propose a transition plan to successfully accomplish this maturation of the technology and/or system.

Metrics

Proposers must clearly define metrics for completing each phase. In particular, evidence-based progression must be clearly demonstrated during the course of Phase 1 for progression to Phase 2 and Phase 3 of the program. Proposers that do not specify evidence-based metrics that allow DARPA to evaluate progress throughout each phase will be considered non-responsive.