Back to the Top
Dear Colleagues,
I'm working with the software WinNonMix (using the FOCE-method at the
moment) and would like to know, whether the minimum objective function
value or the value of -2LL is the parameter to assess the goodness of
fit. I've found in several articles information about the likelihood
ratio test and that for example a decrease of more than 6.6 is
associated with a p-value of 0.01. But this drop of 6.6 as an absolut
number corresponds to -2LL or the minimum objective function value?
This is not consistent in the literature. The users guide of WinNonMix
even says that the parameter estimates are values that minimize an
objective function of the form -2log(likelihood)-nlog(2pi), where
likelihood is an estimate of the likelihood function.
Honestly, I'm confused know, which of these values (minimum objective
function value or -2LL) are associated with this signifcance level and
I hope, somebody can help me answering this question.
I thank you in advance for your assistance.
Susanne Quellmann
Back to the Top
The following message was posted to: PharmPK
WinNonMix like NONMEM reports a number proportional to -2 * log
likelihood as its objective function value. The -n*log(2*pi) constant
can be ignored when two obj function values are compared. The
difference between the objective function values obtained with two
models applied to the same data is commonly used as a statistic to
compare the goodness of fit.
If you want to know the probability of rejecting the null hypothesis
about 2 models using the difference in objective function (DOBJ) as a
test statistic then you need to know the distribution of this test
statistic when the null hypothesis is true. The distribution of DOBJ is
not known in general. Under some rather restrictive assumptions
(particularly assuming normal errors in the model) then it may be
reasonable to *assume* that DOBJ is approximately chi-square
distributed. However, recent empirical tests of this assumed
distribution with some simple cases have shown it cannot be relied upon
to predict the P value associated with a particular DOBJ.
If you really want to know the P value then you will need to determine
it empirically using the randomization test e.g. see
http://wfn.sourceforge.net/wfnrt.htm. This is tedious and often
impractical (especially with WinNonMix which has no support for
batching thousands of model runs which is essential to perform the
randomization test).
Wählby U, Jonsson EN, Karlsson MO. Assessment of the actual
significance levels for covariate effects in NONMEM. Journal of
Pharmacokinetics & Pharmacodynamics 2001;28:23-252
Wählby U, Bouw R, Jonsson EN, Karlsson MO. Assessment of Type I error
rates for the statistical sub-model in NONMEM. Journal of
Pharmacokinetics and Pharmacodynamics 2002;29(3):251-269.
Gobburu JVS, Lawrence J. Application of resampling techniques to
estimate exact significance levels for covariate selection during
nonlinear mixed effects model building: some inferences. Pharmaceutical
Research 2002;19(1):92-98.
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
Zealand
email:n.holford.-a-.auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556
http://www.health.auckland.ac.nz/pharmacology/staff/nholford/
Back to the Top
Dear Nick,
Greetings from another Downunder
Just for my own information, is the Winnonmix algorithm equivalent to
FOCE (I'd always assumed that Winnonmix was the same as FO)
Thanks,
Noel
Noel E Cranswick
Director, Clinical Pharmacology, Royal Children's Hospital
Director, APPRU, Royal Children's Hospital
Associate Professor, University of Melbourne
5th Floor, Main Building
Royal Children's Hospital,
Parkville
Victoria 3052
Australia
Phone: 61-3-9345 6987
Fax: 61-3-9345 5093
Mobile: 0407 512 583
www.appru.com
Back to the Top
The following message was posted to: PharmPK
Noel,
The default estimation method used by WinNonMix is more like FOCE than
FO. The algorithm is not identical but similar. My own evaluation of
WNM vs NONMEM indicated that they have broadly similar properties in
terms of ability to find an objective function minimum and the
associated parameter estimates.
Nick
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
Zealand
email:n.holford.-at-.auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556
http://www.health.auckland.ac.nz/pharmacology/staff/nholford/
Back to the Top
The following message was posted to: PharmPK
Dear Noel:
WinNonMix offers both the FO and conditional first-order (FOCE)
estimation methods. Our implementation for FOCE is a variant of the
Lindstrom-Bates method (Biometrics 46: 673-687). Accordingly,
WinNonMix's FOCE method is theoretically comparable to NONMEM's.
Furthermore, benchmark testing of our software has indicated that,
generally speaking, differences in the numerical results of the two
packages are negligible.
I hope this information is helpful. If there is any way that I may be
of further assistance, please let me know.
With Best Regards,
Mark
Mark R. Lovern, Ph. D.
Director
Technical Pre-Sales and Training
Phone: (919) 852-4607
Mobile (919) 622-2296
FAX: (919) 859-6871
5520 Dillard Drive, Suite 210
Cary, NC 27511
Back to the Top
Dear Susanne Quellmann:
Since you bring up the subject of goodness of fit, and the
calculation of the likelihood function using the FOCE method, it might
be useful to look at the examination of the consistency and efficiency
of population modeling methods which use the FOCE approximation to
compute the likelihood, and those which do not. Exact calculation of
the likelihood function is possible with the nonparametric methods such
as the NPML of Mallet, and the NPEM of Schumitzky, which do not have to
do the integration that is needed with the parametric methods using
such approximations. Go to our web site www.lapk.org, click on New
advances in population modeling, and see the work that Robert Leary
presented at the PAGE meeting in Paris last June. Consistent methods
obtain results that get closer and closer to the true values as more
and more subjects are studied. This is not true of methods using the
FOCE approximation, as he shows there. Because of this, even when the
population parameter distributions are truly Gaussian, the means,
variances, and covariances obtained with the NP methods are more
reliable than those obtained using the FOCE approximation.
Very best regards,
Roger Jelliffe
Roger W. Jelliffe, M.D. Professor of Medicine,
Division of Geriatric Medicine,
Laboratory of Applied Pharmacokinetics,
USC Keck School of Medicine
2250 Alcazar St, Los Angeles CA 90033, USA
Phone (323)442-1300, fax (323)442-1302, email= jelliffe.at.usc.edu
Our web site= http://www.lapk.org
PharmPK Discussion List Archive Index page
Copyright 1995-2010 David W. A. Bourne (david@boomer.org)