I. Myrtveit and E. Stensrud (Norway)
Software cost estimation, arbitrary functionapproximators, case-based reasoning, estimation byanalogy, classification and regression trees, artificialintelligence.
Estimating software development cost with high accuracy
is still a largely unsolved problem. Consequently, there is
ongoing, high activity in this research field; a large
number of different estimation models ranging from
mathematical functions to arbitrary function
approximators (AFA's) have been proposed over the last
20+ years. Unfortunately, the studies do not converge
with respect to the question "which model is best?" when
functions and AFA's are compared. So far, it has not
been understood why this is so. In this empirical study,
we show that this is due to inappropriate validation
methods as far as the validation of AFA's is concerned.
In fact, the de facto validation method, cross-validation
combined with MMRE, will give completely arbitrary
results for AFA's. Obviously, other criteria are called for
in order to appropriately assess the performance of
AFA's. This should be a topic of future research.