This dissertation conducts a series of studies that seek to extend the applicability of empirical methods in the performance comparison of randomized optimization algorithms, introduce new procedures for risk optimization to estimate parameters according to the principles of loss-based estimation, and devise improved confidence intervals for Negative Binomial random variables of high dispersion. Chapter 2 proposes a general framework for the statistical comparison of randomized optimization algorithms based upon multiple hypothesis testing. This method allows for the selection of general performance metrics without imposing parametric assumptions on the data. The comparison framework may be easily adapted to alternative study designs such as relative improvement studies, time-dependent performance comparison, or results based upon multiple objective functions. In the case of time-dependent studies, we propose a data reduction technique to estimate the confidence region across all generations in a more computationally feasible manner. Chapter 3 considers the problem of effectively estimating parameters of the joint distribution of outcome and multivariate explanatory data such as regression functions. We analyze the size of the parameter space of polynomial regression under constraints on the interaction order and the polynomial or variable degree. We demonstrate that such a space may be at least of a doubly exponential size and propose a method of pruning the parameter space by limiting variable interactions. We also devise an evolutionary algorithm for risk optimization that may be used with cross-validation to effectively estimate parameters. We prove that such a procedure asymptotically converges in sample size and generation to the globally optimal estimator within the parameter space. The proposed method is adaptable to alternative parameterizations, modular in its design, and may be tuned to improve performance on the problem at hand. We then investigate its performance through simulation studies and a diabetes data analysis. Chapter 4 seeks to construct improved confidence intervals for Negative Binomial random variables of high dispersion on small samples of data. We demonstrate in this setting that the sample mean will exhibit a slow convergence to the Normal distribution as a function of the sample size, and therefore standard techniques like the Normal approximation and bootstrap will result in confidence intervals that significantly undercover the mean at moderate sample sizes. We postulate and provide empirical evidence for a Chi Square model as an approximate distribution of the sample mean. We propose confidence interval methods based upon this distribution and also Bernsteins inequality, and we demonstrate in simulation studies that these techniques lead to improved coverage in small sample and high dispersion settings. We subsequently examine how Bernstein confidence intervals may be improved through refinements of the choice of upper bound and perform a sensitivity analysis for the simulation results. Finally, we apply these methods to studies arising from the serial analysis of gene expression and traffic flow in a communications network to better elucidate the strengths and weaknesses of all four methods.