The design and development of methods for tracking targets in visual images has developed rapidly in the past decade. However, in practice the design of tracking algorithms is still largely ad-hoc, based on trial and error. As a result, the performance of such algorithms can vary widely based on the properties of the target of interest and the choice of design. The use of spatial sampling kernels on multiple feature spaces has recently emerged as a promising approach to visual target tracking. In particular, it is possible to show that most popular tracking algorithms can be expressed within this framework. As a result, sampling kernels can be viewed as a flexible design space for tracking algorithms. However, in the current approaches, the kernels are placed in an adhoc fashion at the center of the target with a scale equal to the size of the target. This can lead to sub-optimal tracking results. In this paper, we present results pointing toward the design of optimal and approximately optimal target-specific tracking algorithms. The target tracking problem is formulated in terms of an optimization over a family of kernelbased sampling functions. This optimization is solved to produce an optimal target-specific kernel configuration. Experimental results show greatly improved performance over classical template tracking and naive kernel-based tracking.