AdaptiveGridDoc.txt

This is the educational project with main purpose is to develop and test the multidimensional adaptive grid which can be used in Reinforcement Learning area. Additional purpose of this project is to learn and try CUDA programming with Alea GPU.
In typical reinforcement learning scenario an agent takes actions corresponding to current policy and state of an environment.
Adaptive grid is needed to represent a manifold of possible actions of an agent and manifold of states of an environment with ability to adjust grid resolution in areas of interest.
The concept of adaptive refinement of the manifolds of possible states and actions is close to common learning practices (for example, in learning of car driving at a very first stages there is no need for any distinction of severity of bends, instead, the only distinction between left and right turns could be suitable. The more experience and average speeds, the more details about bends and the more precise steering are needed for further improvement of driving skills)

The class AdaptiveGrid is implemented as voxel tree. Each voxel exposes its properties and methods through the IVoxel interface. The maximum number of voxels and shape of the grid are set at creation time of the AdaptiveGrid instance and can't be changed during the liftime of the instance. The voxels are stored in the internal single-dimensional array, which entirely initialized at creation time of the AdaptiveGrid instance. So, during different manipulations with the grid (voxel tessellation or merging), voxels only are changed their properties and no voxel instances are created or deleted. The grid can be processed in multithreaded manner. The methods with names RefineGrid_CGParallel, RefineGrid_FGParallel and others, which can be found in the TestWpf project in the MainWindow.xaml.cs file, represent templates of multithreaded processing and also show how the voxel iterator NextAtLevel works.

The TestWpf project is intended for testing of two-dimensional AdaptiveGrid instances, for estimation of performance of the grid processing methods and for trying some CUDA programming, which in this case was used for bitmaps calculation. The TestWpf has graphical output for visual presentation of the grid and voxels. Several simple abstract models were created for the testing of the grid refinement processes and for fun in some ways.