Despite a large interest in real-world problems, the established benchmarks in the field of evolutionary computation are mostly artificial. We propose to use game optimization problems to form test suites designed to work with the established COCO benchmarking framework. Game optimization problems are real-world problems that are safe, reasonably complex and at the same time practical, as they are relatively fast to compute. We have created four function suites based on two optimization problems previously published in the literature (TopTrumps and MarioGAN). For each of the suites, we implemented multiple instances of several scalable single- and multi-objective functions with different characteristics. Our results prove that game optimization problems are interesting and challenging for evolutionary algorithms.
COBISS.SI-ID: 32547623
We introduce two suites of mixed-integer benchmark problems to be used for analyzing and comparing black-box optimization algorithms. They contain problems of diverse difficulties that are scalable in the number of decision variables. The bbob-mixint suite is designed by partially discretizing the established BBOB (Black- Box Optimization Benchmarking) problems. On the other hand, the bi-objective problems from the bbob-biobj-mixint suite are constructed by using the bbob-mixint functions as their separate objectives. We explain the rationale behind our design decisions and show how to use the suites within the COCO platform. Analyzing two chosen functions in more detail, we also provide some unexpected findings about their properties.
COBISS.SI-ID: 32548135
Surrogate-assisted optimization was developed for handling complex and costly problems, which arise from real-world applications. The main idea behind it is to optimally exhaust the available information to lower the amount of expensive function evaluations thus saving time, resources and the related costs. We outline the existing challenges in this field that include benchmarking, constraint handling, constructing ensembles of surrogates and solving discrete and/or multi-objective optimization problems. We discuss shortcomings of existing techniques, propose suggestions for improvements and give an outlook on promising research directions.
COBISS.SI-ID: 32412967
Until recently, the problems employed for benchmarking optimization algorithms within the COCO platform needed to have continuous variables and known optimal values. In addition, they had to be implemented within the platform (in the C programming language). These restrictions made COCO difficult to use for benchmarking algorithms on real-world problems. This paper describes the adaptations to the COCO platform that facilitate its use on real-world and other problems with integer or mixed-integer variables and unknown optimal values. Evaluation of solutions can now be done with external programs that are interfaced with COCO through socket communication.
COBISS.SI-ID: 32845351
Anytime performance assessment of black-box optimization algorithms assumes that the performance of an algorithm at a specific time does not depend on the total budget of function evaluations at its disposal. It therefore should not be used for benchmarking budget-depending algorithms, i.e., algorithms whose performance depends on the total budget of function evaluations, such as some surrogate-assisted or hybrid algorithms. This paper presents an anytime benchmarking approach suited for budget-depending algorithms. The approach is illustrated on a budget-dependent variant of the Differential Evolution algorithm.
COBISS.SI-ID: 30856231