University of California, Santa Cruz David Draper (last update 28 October 2001)

Table of Contents

(0. Under the pressure of other commitments this web page has fallen somewhat out of date and is in serious need of updating; my apologies.)

1. Recent and upcoming events and news items of possible interest:

  • I am the Chair of the newly forming Department of Applied Mathematics and Statistics (AMS) in the School of Engineering at the University of California, Santa Cruz (I am a statistician).

    AMS is offering a new course next quarter (beginning in the week of 26-30 March 2001) on Bayesian statistical methods and reasoning, which I will be teaching.

    This is a potentially exciting topic to both undergraduate and graduate students with a variety of interests in science and engineering, because

    • uncertainty is pervasive in scientific and engineering problem-solving;
    • the Bayesian approach to the quantification of uncertainty is more flexible and general than other (e.g., relative frequency) approaches. For a long time the Bayesian approach was limited in applications by an inability to perform high-dimensional numerical integrations; but
    • with the advent of powerful computers and new simulation-based techniques over the past 10 years, the computing problem is now solved and there has been a revolution in Bayesian methods and applications.

    Partly because (i) we are new, (ii) I am unfamiliar with how to market such courses, and (iii) I may have set the initial prerequisites too high given that there has not been much teaching of statistics on campus in the past, the pre-enrollment for this course is very low, and it is in danger of being canceled.

    I am going to be very liberal in interpreting the "permission of instructor" prerequisite as a way of inviting all students who might be interested to come to the first few classes -- I will see who shows up and what their backgrounds are, and will tailor the class to the enrollment.

    If you wish to enroll in this class, please contact me by email at draper@ams.ucsc.edu or in person (Baskin Engineering 147) -- I have lots of permission codes.

    A description of the course follows.

    ENG 181: Bayesian Statistics
    Spring 2001
    Instructor: David Draper
    Prerequisites: permission of instructor

    This course will provide an introduction to Bayesian statistical methods for inference and prediction.

    Statistics is the study of uncertainty - how to measure it, and what to do about it. As such, it is of potential interest in many (virtually all?) aspects of science and decision-making. Of the two main ways to quantify uncertainty -- involving relative frequency and subjective (Bayesian) notions of probability -- the second way is more flexible and general, but for a long time the Bayesian approach was limited in applications by an inability to perform high-dimensional numerical integrations. With the advent of powerful computers and new simulation-based techniques over the past 10 years, the computing problem is now solved, and there has been a revolution in Bayesian methods and applications.

    The course will be methodological but will be guided by a series of real-world case studies. The first half of the course will involve symbolic mathematical calculations in the computer package Maple, and statistical analyses and graphics in the package R; contemporary Bayesian computation using the package WinBUGS will feature prominently in the second half.

    The instructor, who has won or been nominated for major teaching awards at four leading universities in the US and England, will survey the background of the initial participants in the course in mathematics and probability to decide how the course should be run for maximal benefit of the participants. He intends to provide a course that will be interesting and profitable for a variety of students at both the undergraduate and graduate levels.

    The week-by-week breakdown of topics to be covered is as follows.

    • Week 1: Probability as quantification of uncertainty about observables. Discrete outcomes; single-parameter problems. Case Study: Hospital-specific prediction of mortality for heart attack patients.
    • Week 2: Exchangeability as a Bayesian concept parallel to frequentist independence. Prior, likelihood, posterior, and predictive distributions.
    • Week 3: Inference and prediction; coherence and calibration. Conjugate analysis. Software: Maple.
    • Week 4: Comparison with frequentist modeling. Continuous outcomes. Gaussian models. Software: R.
    • Week 5: Multiparameter problems. Integrating over nuisance parameters. Case Study: Measurement of physical constants.
    • Week 6: Markov chains. Introduction to Markov Chain Monte Carlo (MCMC) methods.
    • Week 7: Gibbs sampling and Metropolis-Hastings sampling. Software: WinBUGS.
    • Week 8: Hierarchical modeling. Case study: Poisson random-effects modeling: A controlled experiment to assess effectiveness of in-home geriatric assessment.
    • Week 9: Bayesian model diagnostics. Model expansion and cross-validation.
    • Week 10: Bayesian model selection and sensitivity analysis.

    Reading List

    The main textbook for the course will be

    • Gelman A, Carlin JB, Stern HS, Rubin DB (1995). Bayesian Data Analysis. New York: Chapman & Hall.
    Supplementary reading will be taken from
    • Gilks WR, Richardson S, Spiegelhalter DJ (1996). Markov Chain Monte Carlo in Practice. New York: Chapman & Hall.
    and from
    • Draper D (2001). Bayesian Hierarchical Modeling. New York: Springer-Verlag (forthcoming).

    Evaluation

    There will be homework assignments (more like small take-home tests) given out in weeks 2, 4, 6, and 8 and due one week later; these will blend paper-and-pen, symbolic computing, statistical and MCMC calculations. A take-home final exam will be assigned in week 9 and due at the end of the final examination period.

  • I gave a tutorial on Bayesian hierarchical modeling at the ISBA 2000 meeting in Crete in May 2000. A revised PostScript version of this tutorial - which fills in most of the blank pages in the previous version - is available for downloading here. There are still a few blank pages in this version; these are just placeholders for some screen shots from using WinBUGS to fit the models whose analysis I illustrate (I couldn't figure out how to incorporate the screen shots into the PostScript document). If you have any comments on the tutorial I would be interested to hear them at draper@ams.ucsc.edu.
  • I gave a one-day short course on Bayesian hierarchical modeling at the Interface 2000 meeting (the 32nd symposium on the interface between computer science and statistics) in New Orleans in April 2000. More details about the meeting are available here, and a complete description of the course may be found here. The course was co-sponsored by LearnStat, the continuing education branch of the American Statistical Association.

    At this year's Interface 2001 meeting in southern California, I will give a two-day short course on Bayesian hierarchical modeling, with the first day on introductory and intermediate topics and the second day on advanced issues. This is scheduled to take place on (Mon-tue) 11-12 June 2001.

  • Drafts of articles and book chapters recently finished, available for downloading:
    • A comparison of Bayesian and likelihood-based methods for fitting multilevel models (PostScript format) (with Browne WJ; October 2000): submitted (a substantially revised new version of the previous August 1999 paper). (We use simulation studies, whose design is realistic for educational and medical research, to compare Bayesian and likelihood-based methods for fitting variance-components (VC) and random-effects logistic regression (RELR) models. The likelihood (and approximate likelihood) approaches we examine are based on the methods most widely used in current applied multilevel analyses: maximum likelihood (ML) and restricted ML (REML) for Gaussian outcomes, and marginal and penalised quasi-likelihood (MQL and PQL) for Bernoulli outcomes. Our Bayesian methods use Markov chain Monte Carlo (MCMC) estimation, with adaptive hybrid Metropolis-Gibbs sampling for RELR models, and several diffuse prior distributions (inverse gamma and uniform priors for variance components). For evaluation criteria we consider bias of point estimates and nominal versus actual coverage of interval estimates. In two-level VC models we find that (a) both likelihood-based and Bayesian approaches can be made to produce approximately unbiased estimates, although the automatic manner in which REML achieves this is an advantage, but (b) both approaches had difficulty achieving nominal coverage in small samples and with extreme values of the variance parameters (as measured by the ratio of variances at levels 1 and 2). With three-level RELR models we find that (c) quasi-likelihood methods for estimating random-effects variances performed badly with respect to bias and coverage in the example we simulated, and (d) Bayesian diffuse-prior intervals lead to well-calibrated point and interval RELR estimates. Given that the likelihood-based methods we study are considerably faster computationally than MCMC and that a number of models are typically fit during the model exploration phase of a multilevel study, one possible analytic strategy suggested by our results is a hybrid of likelihood-based and Bayesian methods, with (i) REML and quasi-likelihood estimation (for their computational speed) during model exploration and (ii) diffuse-prior Bayesian estimation using MCMC to produce final inferential results. Other analytic strategies based on less approximate likelihood methods are also possible but would benefit from further study of the type summarised here.).)
    • Bayesian and likelihood methods for fitting multilevel models with complex level-1 variation (Browne WJ, Draper D, Goldstein H, Rasbash J; July 2000; PostScript, 404K; prints 21 pages): submitted. (In multilevel modeling it is common practice to assume constant variance at level 1 across individuals. In this paper we consider situations where the level-1 variance depends on predictor variables. We examine two cases using a dataset from educational research; in the first case the variance at level 1 of a test score depends on a continuous "intake score" predictor, and in the second case the variance is assumed to differ according to gender. We contrast two maximum-likelihood methods based on iterative generalized least squares with two MCMC methods based on adaptive hybrid versions of the Metropolis-Hastings (MH) algorithm, and we use two simulation experiments to compare these four methods. We find that all four approaches have good repeated-sampling behavior in the classes of models we simulate. We conclude by contrasting raw- and log-scale formulations of the level-1 variance function, and we find that adaptive MH sampling is considerably more efficient than adaptive rejection sampling when the heteroscedasticity is modeled polynomially on the log scale.)
    • A Case Study of Stochastic Optimization in Health Policy: Problem Formulation and Preliminary Results (with Fouskakis D; May 2000; PostScript, 582K; prints 22 pages): Journal of Global Optimization, forthcoming. (We use Bayesian decision theory to address a variable selection problem arising in attempts to indirectly measure the quality of hospital care, by comparing observed mortality rates to expected values based on patient sickness at admission. Our method weighs data collection costs against predictive accuracy to find an optimal subset of the available admission sickness variables. The approach involves maximizing expected utility across possible subsets, using Monte Carlo methods based on random division of the available data into N modeling and validation splits to approximate the expectation. After exploring the geometry of the solution space, we compare a variety of stochastic optimization methods - including genetic algorithms (GA), simulated annealing (SA), threshold acceptance (TA), messy simulated annealing (MSA), and tabu search (TS) - on their performance in finding good subsets of variables, and we clarify the role of N in the optimization. Preliminary results indicate that TS is somewhat better than TA and SA in this problem, with MSA and GA well behind the other three methods. Sensitivity analysis reveals broad stability of our conclusions.)
    • Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models (with Browne WJ; December 1999; PostScript, 525K): Computational Statistics, forthcoming. (Explores Bayesian and likelihood fitting methods, in terms of validity of conclusions, in two-level random-slopes regression (RSR) models, and compares several Bayesian fitting methods based on Markov chain Monte Carlo, in terms of computational efficiency, in random-effects logistic regression (RELR) models.)
    • Model uncertainty yes, discrete model averaging maybe (September 1999): comment on "Bayesian model averaging: a tutorial," by Hoeting JA, Madigan D, Raftery AE, Volinsky CT, Statistical Science, forthcoming. (Argues that variable selection uncertainty in generalized linear models should be dealt with in a continuous manner via hierarchical modeling rather than through discrete model averaging, and advocates the use of expected utility maximization as a basis for model choice.)
    • Book review (with Fouskakis D; May 1999) of "Tabu Search," by Glover F, Laguna M, Journal of the Royal Statistical Society, Series D (The Statistician), forthcoming. (Discusses the only book-length treatment of tabu search, a good stochastic optimization method developed by people in operations research and not well known yet by statisticians.)
    • Scenario and parametric sensitivity and uncertainty analyses in nuclear waste disposal risk assessment: the case of GESAMAC (with Saltelli A, Tarantola S, Prado P; revised May 2000): Chapter 13 in Mathematical and Statistical Methods for Sensitivity Analysis (Saltelli A, Chan K, Scott M, eds.), New York: Wiley (2000), 275-292. (Shows that variance-based sensitivity analyses are not fully adequate in determining the factors most responsible for high radiologic doses arising from the failure of underground storage facilities for nuclear waste, and that about 30% of the overall predictive uncertainty for log dose arises from uncertainty about the scenario describing how the facility will fail -- a source of uncertainty previously largely ignored or treated qualitatively. Also explores the use of projection pursuit regression in sensitivity analysis.)
    • Sampling errors under non-probability sampling (with Bowater R; January 1999): Chapter 4 in Model Quality Reports in Business Statistics: Theory and Methods for Quality Evaluation, by Bowater R, Chambers C, Davies P, Draper D, Skinner C, Smith P. Luxembourg: Eurostat. (Book-length report produced by team consisting of people from the Office for National Statistics (UK), Statistics Sweden, and the Universities of Bath and Southampton; in this chapter we review biases arising from voluntary sampling, judgmental sampling, quota sampling, and cut-off sampling, and make recommendations on how to assess and minimize such biases.)
    • Model assumption errors in survey sampling (with Bowater R; January 1999): Chapter 9 in Model Quality Reports in Business Statistics: Theory and Methods for Quality Evaluation, by Bowater R, Chambers C, Davies P, Draper D, Skinner C, Smith P. Luxembourg: Eurostat. (See item above; in this chapter we review model assumption errors as they arise in the construction of index formulae, bench-marking, seasonal adjustment, cut-off sampling, small-area estimation, and non-ignorable nonresponse, and make recommendations on how to assess and minimize such errors.)
    • Scenario and parametric uncertainty in GESAMAC: A methodological study in nuclear waste disposal risk assessment (with Pereira A, Prado P, Saltelli A, Cheal R, Eguilior S, Mendes B, Tarantola S; November 1998): Computer Physics Communications, forthcoming. (Companion piece to the first item above, with similar results and methods on a different source of radioactive decay.)
  • Draft chapters of Bayesian Hierarchical Modeling (text and computer programs) now available for downloading
  • Bayesian MCMC multilevel modeling workshops (6 April and 29 October 1998)
  • Bayesian hierarchical modeling short courses (August 1998)
  • International workshop on stochastic model building and variable selection (Duke University, 9-10 October 1997)
  • RSS half-day meeting on design and analysis of complex sample surveys (14 May 1997)
2. Research and teaching
  • Contact information
  • Biographical sketch
  • Statistical philosophy and outlook
  • Current research interests (lots of papers available for downloading)
  • Funded research projects
  • Some thoughts on statistics teaching
  • Present and past postdocs and PhD/MSc students
  • PhD projects on offer

3. Personal information

1. Recent and Upcoming Events of Possible Interest

1.1 Draft chapters of Bayesian Hierarchical Modeling
(text and computer programs) now available

I am working on a research monograph on Bayesian Hierarchical Modeling. The second draft of the first two chapters (together with a preface, an appendix with computer programs, and references) is now available (in PostScript format) here for downloading and free use. (Number of people worldwide who have downloaded the text and programs so far: about 1,250.)

The intended audience for the book is methodological and applied statisticians who wish to learn (more) about the formulation and fitting of hierarchical (multilevel) models from the Bayesian point of view.

An understanding of probability at the level typically required for a master's degree in statistics would provide ample mathematical background for reading the book. I have taught subsets of the draft material successfully to groups including British final-year undergraduates, American PhD students, and PhD-level researchers enrolled in short courses (including an award-winning course at the Anaheim Joint Statistical Meetings in 1997), and the book has also proven useful for self-study by researchers and graduate students in a variety of disciplines (including statistics). No previous experience with Bayesian methods is needed -- all relevant ideas are covered in a self-contained fashion.

The draft manuscript PostScript file is about 1.5Mb, and when printed you get 183 pages: Contents and Preface (pp. i-xiv), Chapter 1 (pp. 1-46), Chapter 2 (pp. 47-122), some placeholder stuff you can avoid printing (pp. 123-138, 169), Appendix 2 (pp. 139-160), and References (pp. 161-168).

The first chapter (46 pages) provides a standalone introduction to Bayesian modeling in the context of two case studies, and the second chapter (76 pages) offers an in-depth look at Markov Chain Monte Carlo (MCMC) methods from first principles, also based on two case studies. The writing style is informal; the main text is not very mathy, but each chapter is supplemented by extensive endnotes giving additional formalism and details.

Appendix 2 contains S+, BUGS, Maple, and C programs for conducting the analyses in Chapter 2; these programs are also available for downloading here as a 36Kb text file. Chapter 2, when combined with computer work based on the code supplied here, might make a nice MCMC tutorial to supplement any other coverage you may have on this topic, and Chapter 1 might serve as a gentle introduction to Bayes for advanced undergraduates or beginning grad students.

The only things I ask in return for the free use of these materials in your teaching are (a) that you email me giving details (class or tutorial size and name, level of students) of both (i) when and where you have used the book and programs, and (ii) any errors or problems you encounter, or other comments you wish to pass on; and (b) that - if you like this material - you consider buying a copy of the finished book, which (I hope) will be published within the next 12 months!

1.2 Bayesian MCMC multilevel modeling workshops
(6 April and 29 October 1998)

One-day workshops on MCMC methods in multilevel modeling were given by David Draper (University of Bath) and Bill Browne (Institute of Education, University of London) on April 6 and October 29, 1998 at the Institute of Education, using the new Windows version of MLn (MLwiN), which has recently been released. The workshops were offered in conjunction with the Multilevel Modelling Project, led by Harvey Goldstein.

The workshops combined methodology discussion with real-time hands-on computing experience, and there was also an opportunity for participants to submit their data sets for inclusion as case studies for interactive analysis in the afternoon session.

The intended audience was multilevel/hierarchical modelers (of varying levels of experience, from not much to quite a lot) who wish to learn about Markov Chain Monte Carlo (MCMC) Bayesian methods and their implementation in MLwiN, and people interested in learning about the new interactive model specification, fitting, and diagnostic capabilities of the new Windows version of MLn also found the workshops worthwhile. Bill Browne and I are the co-developers of the MCMC functionality in MLwiN.

We are interested in giving future workshops of this type, and are open to suggestions on location and timing. If you would like to discuss this possibility, or for further details, please email Bill Browne, or phone me at +44 (0) 1225 826222. The MLwiN software would be available for a discount as part of coming to the workshop.

A tentative program for the day is as follows:

  • 9.00 - 10.30: Introduction to multilevel models, including Gaussian and generalized linear multilevel models; hands-on computer work
  • 10.30 - 11.00: Tea, informal discussion
  • 11.00 - 12.15: Introduction to Bayesian inference and prediction
  • 12.15 - 1.30: Lunch, informal discussion
  • 1.30 - 3.00: Markov Chain Monte Carlo (MCMC) estimation and diagnostics; hands-on computer work
  • 3.00 - 3.30: Coffee, informal discussion
  • 3.30 - 5.00: Model formulation, diagnostics, and elaboration; hands-on computer work, including analysis of participants' data
As the tentative program indicates, the workshop includes lectures on the basics of multilevel models, Bayesian inference, and MCMC methods, together with some real-time hands-on computing sessions using the MLwiN package. There is also an opportunity for participants to submit their data sets for inclusion as case studies for interactive analysis in the afternoon session. The first two chapters of my book on hierarchical modeling (see 1.1 above) -- on Bayesian modeling and MCMC -- are now available in draft form (with associated computer code in S+, C, and Maple), and are distributed as part of the workshop.

The cost of the workshop would be as follows:

  • Workshop only: 250 pounds (150 pounds academic)
  • Workshop + 1 copy of MLwiN: 700 pounds (420 pounds academic)
  • Workshop + upgrade from MLn (DOS): 300 pounds (180 pounds academic)
Anyone who has already purchased MLn (DOS) after October 1st 1997 would receive the upgrade to MLwiN for free.

1.3 Bayesian hierarchical modeling short courses

I have recently given two one-day short courses on Bayesian hierarchical modeling, based on the book described in 1.1 above, at the Dallas Joint Statistical Meetings in August 1998, through the American Statistical Association (ASA) Continuing Education program.

The first day was an invited short course, offered at an introductory/intermediate level -- it essentially repeated the course I gave in Anaheim in 1997, which won an ASA Excellence in Continuing Education award. No previous exposure to Bayesian methods was needed in this first course -- all ideas were covered in a self-contained fashion. Topics for the first course included (1) an introduction to Bayesian modeling, (2) MCMC methods from scratch, (3) formulation of hierarchical models (HMs) based on the scientific/decision-making context, and (4) diagnostics for HMs.

The second day covered more advanced topics, including (1) random-effects and mixed models, (2) Bayesian nonparametric inference with Polya trees, and (3) HMs as an approach to model selection and dealing with model uncertainty. Each day can be taken in a standalone fashion, or people can come to both days if they want to.

I am interested in giving these short courses again in the future. If you would like to suggest a time, place, and audience, please email me.



1.4 INTERNATIONAL WORKSHOP
ON STOCHASTIC MODEL BUILDING
AND VARIABLE SELECTION

Duke University, Durham NC
October 9 and 10, 1997

A Workshop whose goal was bringing together researchers interested in novel approaches to computer-based and/or simulation-based aids to model building.

The program of the Workshop included both talks and poster presentations. For lists of the speakers and participants, practical details, and other information see this web page; if you have questions please email Giovanni Parmigiani or me.

1.5 RSS half-day meeting on
design and analysis of complex sample surveys

Date and time: 14 May 1997, 2-6.50pm.

Place: headquarters of the Royal Statistical Society (RSS), 12 Errol Street, London EC1Y 8LX England (voice +44-171-638-8998, fax +44-171-256-7598, email rss@rss.org.uk).

Four papers (available for downloading) by international teams of leading researchers in survey sampling, together with invited and contributed discussion and rejoinder. For more information see this web page or email me.

Bath logo 2. David Draper

2.1 Contact Information

I am a Professor in, and Head of, the Statistics Group in the Department of Mathematical Sciences. My phone number is +44-1225-826222; fax is +44-1225-826492; email is d.draper@maths.bath.ac.uk. My postal address is Statistics Group, Department of Mathematical Sciences, University of Bath, Claverton Down, Bath BA2 7AY, England.

2.2 Biographical Sketch

Before coming to Bath I studied for a BSc in mathematics at the University of North Carolina/Chapel Hill (1970-74) and a PhD in statistics at the University of California/Berkeley (1975-81); worked at IBM in New York (1974-75); and taught and did research at the University of Chicago (1981-84), the RAND Corporation (1984-91), and UCLA (1991-93), with brief sabbatic and consultant stints at the University of Washington (1986) and AT&T Bell Labs (1987). Since arriving in Bath I have also done some teaching at the University of Neuchâtel in Switzerland, and given short courses on Bayesian hierarchical modeling at the joint meetings of the American Statistical Association and the Institute of Mathematical Statistics.

2.3 Statistical Philosophy and Outlook

Philosophically I am some kind of de Finetti-style Bayesian, meaning that for me

  • prediction of observables is more fundamental than inference about unobservables, and
  • (conditional) exchangeability judgments are fundamental to predictive modeling.
Sure, you end up thinking about lots of unobservable parameters in this approach, but they don't come first -- they arise from the use of de Finetti's theorem to pass from exchangeability to conditional independence.

To me it's OK to supplement contextual information from experts with data analysis in forming your exchangeability judgments, as long as you keep yourself honest by not using the data twice in the process.

In practice for me this often means employing predictive calibration: holding out some of the data from the modeling and seeing where the observed outcomes in the held-out data fall in their respective model-based predictive distributions. If this kind of predictive calibration fails, then you have to go back and change the model (which includes the possibility of changing the ``prior'') until you are well-calibrated.

I see this as a kind of fusion of the best of Bayesian and non-Bayesian reasoning: (1) Bayes by itself (when done right) guarantees internal but not (2) external consistency, which involves asking inherently frequentist questions (how often do my predictive intervals include the observed outcomes?).

This philosophy has implications both in research and teaching, which I am currently refining. If you disagree with the views above and would like to talk about it, please email me.

2.4 Current Research Interests

  • Bayesian inference and prediction (e.g., Draper D (1996), Utility, sensitivity analysis, and cross-validation in Bayesian model-checking. Discussion of ``Posterior predictive assessment of model fitness via realized discrepancies," by A Gelman et al., Statistica Sinica, 6, 28-35; Draper D and Madigan D (1997), The scientific value of Bayesian statistical methods and outlook, IEEE Expert, Trends and Controversies department, October/December issue; and 2 other publications);
  • Model uncertainty, and exploration of the mapping from statistical assumptions to conclusions (e.g., Draper D (1988), Statistical Science, 3, 239-271; Draper D (1995), Assessment and propagation of model uncertainty (with discussion), Journal of the Royal Statistical Society Series B, 57, 45--97; Draper D (1997), On the relationship between model uncertainty and inferential/predictive uncertainty, under revision for Biometrika; Draper D (1997), Model uncertainty in stochastic and ``deterministic'' systems, Proceedings of the 12th International Workshop on Statistical Modeling, Biel, July 1997, Schriftenreihe der Osterreichischen Statistichen Gesellschaft, 5, 43-59; and 2 other publications);
  • Theory of data analysis (e.g., Draper D (1987), On exchangeability judgments in predictive modeling, and the role of data in statistical research, Statistical Science, 2, 454-461 (discussion of ``Prediction of Future Observations in Growth Curve Models,'' by CR Rao); Hadorn D et al. (1992), Cross-validation performance of patient mortality prediction models, Statistics in Medicine, 11, 475-489; Draper D et al. (1993), Exchangeability and data analysis (with discussion), Journal of the Royal Statistical Society Series A, 156, 9-37; Greenland S, Draper D (1997), Exchangeability. Entry in Encyclopedia of Biostatistics. Armitage P, Colton T (eds). London: Wiley; Draper D (1998), Discussion of ``Some statistical heresies'' by JK Lindsey, The Statistician, forthcoming; and 1 other publication);
  • Hierarchical modeling (e.g., Draper D et al. (1993), Combining Information: Statistical Issues and Opportunities for Research, Contemporary Statistics Series, No. 1, American Statistical Association, Alexandria, VA; Draper D (1995), Inference and hierarchical modeling in the social sciences (with discussion), Journal of Educational and Behavioral Statistics, 20, 115-147, 233-239; and 3 other publications);
  • Causal inference (e.g., Draper D and Cheal R (1997), Causal inference via Markov Chain Monte Carlo (in preparation));
  • Sample surveys (e.g., Bayesian analysis of finite-population survey data using Markov Chain Monte Carlo. Closing discussion, Half-Day Meeting on Design and Analysis of Complex Sample Surveys, Journal of the Royal Statistical Society Series B, 60, 96--98);
  • Markov Chain Monte Carlo (MCMC) methods (e.g., Cheal et al. (1997), MCMC methods for inference on family trees (in preparation); Goldstein H et al. (1997), A User's Guide to MLn for Windows (MLwiN), Version 1.0b, London: Institute of Education); and
  • Applications of statistical methods to the
    • Social sciences (e.g., Steiner A et al. (1996), Gerontologist, 36, 54-62),
    • Medical and health sciences, particularly quality of care in health policy (e.g., Dubois R et al. (1987), New England Journal of Medicine, 317, 1674-1680; Daley J et al. (1988), Journal of the American Medical Association, 260, 3611-3616; Draper D et al. (1990), Journal of the American Medical Association, 264, 1956-1961; Swezey R et al. (1997), Journal of Rheumatology; and 11 other publications), and
    • Biological and environmental sciences (e.g., Cheal et al. (1997), Inference on founder allele frequencies in the Przewalski horse pedigree (in preparation); Draper (1997), Model uncertainty in stochastic and ``deterministic" systems, Proceedings of the 12th International Workshop on Statistical Modeling, Biel, July 1997, Schriftenreihe der Osterreichischen Statistichen Gesellschaft, 5, 43-59; Draper et al. (1998), Scenario and parametric uncertainty in GESAMAC: A methodological study in nuclear waste disposal risk assessment, Computer Physics Communications, forthcoming).
If you have written something which you've made public on one or more of these topics and you are interested in a dialogue, please email me with details on how to get a copy, and I will try to send you comments.

2.5 Funded Research Projects

I have just finished working with Dr. Ryan Cheal (Bath) and partners at the Environmental Institute, Joint Research Center (Ispra, Italy), CIEMAT (Madrid, Spain), and the University of Stockholm (Sweden) on GESAMAC, a three-year EC-funded environmental project exploring the likely effects on the geosphere from possible future failure of underground containment vessels for spent nuclear fuel.

Ryan and I were helping to quantify all relevant sources of uncertainty (from model input scenarios, model structural assumptions, model parametric variability, and predictive inaccuracy) in forecasts of radiologic dose arising from containment vessel failure. The EC has made the working documents from this project confidential, but two papers now in the open literature are available: Draper (1997), Model uncertainty in stochastic and ``deterministic" systems, Proceedings of the 12th International Workshop on Statistical Modeling, Biel, July 1997, Schriftenreihe der Osterreichischen Statistichen Gesellschaft, 5, 43-59; and Draper et al. (1998), Scenario and parametric uncertainty in GESAMAC: A methodological study in nuclear waste disposal risk assessment, Computer Physics Communications, forthcoming.

If you have interests in this area I would like to start a dialogue with you; please email me.

I have also just finished working with Dr. Russell Bowater and partners at the Office for National Statistics, the Department of Social Statistics at the University of Southampton, and Statistics Sweden on a one-year project funded by Eurostat, to develop and test methodology to better account for all sources of uncertainty in complex business surveys of the type routinely undertaken by EC member countries.

We reviewed and tested methodologies for

  • estimating and adjusting for bias arising from non-probability sampling, and
  • assessing uncertainty arising from incorrect modeling assumptions in the survey sampling context.
The main output of this work is an extensive report, to be published by Eurostat, giving methodology and best practice in the reporting of business sample surveys; this should be available early in 1999. If you have an interest in these areas and would like to start a discussion by email, please write to me.

2.6 Some Thoughts on Statistics Teaching

I also have a basic interest in the teaching of statistics at the BSc, MSc and PhD levels. I have taught at Berkeley, Chicago, RAND, Seattle, UCLA, Bath, Neuchâtel, and the American Joint Statistical Meetings on introductory statistics, design of experiments, sample surveys, multivariate methods, computationally intensive inference, linear models, statistical modeling, Bayesian inference and prediction, and Bayesian hierarchical modeling.

I believe that three principles should govern the teaching of statistics at all levels:

  • It is important to tell the Bayesian and frequentist stories side by side, so that people can clearly see the strengths and weaknesses of each approach and can thereby create their own personal combination of what is good in both approaches. There is a lot of silliness in the frequentist approach, but being Bayesian is no guarantee of getting the right answer, either.

    What works for me is (a) to reason in a Bayesian way when formulating my inferences and predictions and (b) to reason in a frequentist way when evaluating their quality, through predictive calibration (see the section on philosophy above).

    There are certainly other ways to look for the best in Bayes and non-Bayes; I am sure you have your own (strong) views on the subject, and I would be interested to hear them.

  • More emphasis should be placed on prediction than in current treatments of statistics, which almost always focus almost exclusively on inference. Both science and decision-making are inherently predictive at heart: good scientific theories make testable (and accurate) predictions, and decision theory is all about making predictions about the future under different scenarios and choosing your favorite future.

    Many inferential questions can usefully be rephrased in predictive terms, e.g., if you are a physician the key medical question is often not (whether treatment A is better on average than treatment B in some population) but (how much different the outcome would be under the two treatments for the patient in front of you). Given all of this, why do we spend so little teaching time on prediction?

  • Each unit of material should begin with (1) a scientific or decision-making case study, with sufficient contextual details for the real-world problem to be clearly in focus. Then (2) the statistical methods that are the point of this unit can be developed in the context of the case study, after which (3) these methods can be applied to solve the real-world problem in (1) that prompted the inquiry in the first place. After this the unit can be concluded with (4) an investigation of the general properties of the methods developed in (2).

    This four-step approach echoes the process by which the methods were originally developed, which encourages people to see how ideas are discovered in the first place. It is especially good to cover step (2) in an interactive way, asking people to help suggest ideas for what to do next and exploring rather than condemning dead-ends, because in practice the discovery process itself often proceeds by learning what is wrong with each of a series of partial failures.

I am working at present on two books - a monograph on Bayesian hierarchical modeling (see 1.1 above) and an introductory text - that try to follow these three principles, at least roughly. If your views on the best way to teach statistics differ and you would like to talk about it, please email me.

2.7 Present and past postdocs and PhD/MSc students
  • Dr. Russell Bowater is a postdoc who finished his PhD with Bernard Silverman at Bristol in 1997, on MCMC in a nonstandard spatial statistics application in biology. Russell and I have just finished working on the Eurostat complex sample surveys project described above.
  • Bill Browne (bwjsmsr@ioe.ac.uk) recently finished his PhD on Applying MCMC methods to multi-level models, including putting MCMC into the popular package MLwiN for fitting hierarchical models. His dissertation was nominated for the 1998 Leonard Jimmy Savage Award for best Bayesian PhD dissertation in the world. Earlier we worked together on his MSc dissertation (Topics in hierarchical modeling), which was the recipient of the James Duthie Prize in 1995. In October 1998 Bill started a postdoc with Harvey Goldstein at the Institute of Education of the University of London;
  • Dr. Ryan Cheal worked with me on his PhD (Markov chain Monte Carlo methods for inference on family trees), finishing in 1997. He was until recently a postdoc on the GESAMAC project (see 2.5 above).
  • Dimitris Fouskakis is a PhD student working with me on Stochastic optimization for cost-effective quality assessment in health, a project using Bayesian modeling and utility analysis to construct optimal scales for measuring inputs (such as patient sickness at admission to the hospital) in league-table (input-output) quality assessment. Previously we worked together on his MSc dissertation, Variable selection via hierarchical modeling and utility, which was awarded distinction in 1996. Dimitris has been short-listed for the 1999 Ede and Ravenscroft Research Prize at the University of Bath.
  • Mark Gittoes has just started a PhD with me in September 1998 on Hierarchical modeling for quality assessment in health and education.
  • Daphne Kounali did her MSc dissertation work with me on Cardiac mortality and dietary risk factors: Survival analysis with time-varying covariates, finishing in 1998. She is now a medical statistician in the Research and Development Unit at the Salford Royal Hospitals NHS Trust.
  • Callum McKail worked with me in 1997 on his MSc dissertation, Fixing the broken bootstrap: Bayesian inference with skewed and long-tailed data. He is now with a software development company in the London area.
  • Kristi Raube did her PhD with me at the RAND Graduate School of Policy Studies, finishing in 1991; her dissertation was on Health and social support in the elderly. She is now Acting Director of the Center for Health Administration Studies at the University of Chicago.

2.8 PhD Projects on Offer

I am currently looking for PhD students to work with me on the following topics:

  • Assessment and propagation of model uncertainty. Models formalizing facts and assumptions about known and unknown quantities are central to statistical inference and the prediction of future observables. It is common practice to search for a reasonable model and then settle on a single choice, ignoring the model uncertainty uncovered in the search. Conditioning on a single model, when others with different predictive consequences are also plausible, under-propagates an important component of uncertainty, leading to predictive uncertainty assessments that, in retrospect, are often seen to be too small. This has direct consequences in recommending actions that do not hedge sufficiently against uncertainty.

    The Bayesian solution is essentially to deal with model uncertainty in the way that a nuisance parameter would be treated, by integrating over it as in the following three-step program (Draper D (1995), Assessment and propagation of model uncertainty (with discussion), Journal of the Royal Statistical Society Series B, 57, 45--97):

    • Put a prior distribution on model space and update to a posterior on the set of possible models given the data;
    • Compute a predictive distribution based on each model with nonzero posterior probability; and
    • Produce a composite posterior predictive distribution that combines model-specific predictive distributions weighted by their posterior probabilities on model space.
    Current projects in this area include an exploration of the role bootstrapping and Markov Chain Monte Carlo methods can play in approximating posterior model probabilities in complex problems. For example, bootstrapping the modeling process -- creating bootstrap copies of a data set, conducting parallel modeling exercises on each copy (including outlier deletion and variable selection and transformation), and combining within-copy and between-copy uncertainty assessments to produce better-calibrated predictions -- is a largely untested but promising way to use the bootstrap to approximate posterior model probabilities that reflect the realistic complexity of applied data analysis.

  • Input-output analysis for quality assessment in health and education. Two major societal institutions that have recently begun to look more closely at the quality with which they carry out their mandates are hospitals and schools. In the case of hospitals, a number of factors relevant to quality assessment have been identified, including the processes of care (what health professionals actually do on behalf of patients), the outcomes of care (what happens to patients as a result of the processes they receive), and patient sickness at admission (since hospitals differ widely in the severity of illness of their patients, and any examination of their health outcomes must bear this in mind). Similar concepts apply to schools.

    In both instances process is difficult and expensive to measure, so interest has begun to focus on a less costly input-output approach to quality assessment, in which institutional outcomes are compared after adjusting for differences in inputs. In the case of hospitals this may take the form of a contrast between observed and expected mortality rates, given how sick patients are when they arrive at the hospital; in schools value-added computations examining A-level results (from standardized tests taken in the last year of high school) after accounting for GCSE scores (from another set of standardized tests taken one year earlier) have begun to emerge. Bayesian hierarchical modeling that is explicitly tailored to the multilevel structure of the data (patients nested within hospitals, students within classrooms within schools) plays a central role in this approach to quality assessment.

    Projects available in this area include (a) the solution of technical problems posed by the hierarchical modeling of massive data sets and (b) explicit attempts to guide public policy by using such modeling to suggest optimal data collection strategies and optimal allocation of resources between input-output and process-only quality assessments.

  • Causal inference via Markov chain Monte Carlo. A major challenge in the analysis of observational studies is the assessment of the likely influence of unmeasured potential confounding factors on the estimated effect of the principal causal factor of interest. Two leading classes of models for such data are selection models (developed by the economist J Heckman) and counterfactual models (introduced by J Neyman and developed by D Rubin). The fully Bayesian analysis of such models with Markov chain Monte Carlo methods poses several technical problems, including multi-modality of the posterior distribution and extremely high serial correlation of the Monte Carlo draws. In addition to solving these problems, this project involves the specification of appropriate informative prior distributions for key quantities not well addressed by the data, and a detailed comparison - in theory, simulations, and case studies - between the selection and counterfactual approaches.

  • Developing a general-purpose Metropolis-Hastings engine. The program BUGS, created by the MRC Biostatistics Unit in Cambridge, is an elegant and rather general-purpose environment within which to perform approximate Bayesian inference with Gibbs sampling. An analogous Metropolis-Hastings engine, suitable for an even wider variety of inferential and predictive situations than that addressed by BUGS, has not yet been developed. This project would explore several strategies for constructing such an engine, including adaptive selection of multivariate normal proposal distributions after appropriate parameter transformation.

  • Nonparametric Bayesian analysis. Using de Finetti's theorem to approach, from first principles, the construction of well-calibrated predictive and inferential distributions for observable and unobservable quantities, respectively, requires placing a prior distribution on the set of all possible CDFs, a problem that has until recently had no fully satisfying solution. Even nonparametric bootstrap confidence intervals, which can be regarded as crude approximations to posterior distribution summaries of particular interest, perform surprisingly poorly with fairly large samples of long-tailed data, because the empirical CDF has nothing to say about the tails of the distribution beyond the largest observation. A similar problem arises in Bayesian analyses of generalized linear models, when attempts to honestly assess uncertainty about the link function require placing a prior on the set of all possible regression surfaces.

    This project involves the theory and application of MCMC in the context of Polya trees - and other approaches to non-parametric Bayesian inference - in a way that is responsive to actual applied prior knowledge on, e.g., unimodality, tail behavior, and moments of CDFs (on the one hand) and monotonicity and smoothness of regression surfaces (on the other), in the context of theory, simulations, and real case studies. I have recently used Polya trees to solve a consulting problem for AEA Technologies in risk assessment arising from nuclear waste disposal, and I am eager to explore the practical limits of this modeling strategy.

  • Bayesian predictive validation as an approach to solving the problem posed by Cromwell's Rule. Cromwell's Rule (Lindley 1972) reminds us that, in the Bayesian formulation to statistical modeling, anything with zero prior probability must also have zero posterior probability, no matter how the data came out. This poses a dilemma for practical Bayesian modeling: we must, on grounds of feasibility, place zero initial prior probability on vast regions of the space of all possible models for the observed data, and yet once the data arrive we may well regret having marked as impossible in the prior various features of the data that clearly are not impossible, because they actually occurred. Thus there is a need in Bayesian modeling (just as with other approaches) to update prior guesses about model structure in light of the data, but without cheating by using the data twice.

    One way out of this difficulty involves the sort of Bayesian nonparametric analyses described in the previous project. In this project another approach, out-of-sample predictive validation, will be used to overcome the central problem posed by Cromwell's Rule. The idea is to cross-validate the modeling process - by setting aside some data and using a range of models fitted to another subset of the data to predict the set-aside observations - but in such a way as to obtain an honest estimate of predictive accuracy of the composite modeling process. Theory, simulations, and case studies will be used to explore the strengths and limitations of this approach.

3. Personal Information

My wife is Dr. Andrea Steiner, a Senior Lecturer in gerontology and health policy analysis in Social Sciences and Geriatric Medicine at the University of Southampton. We live in an old stone house in Limpley Stoke, a nice village on the river Avon near Bath, with a canal, some good pubs, and some excellent hill-walking nearby. I now know a lot more about real ale than I did six years ago.