At the moment, the only way to skip around a video without the pointer is to skip around manually at fixed intervals with the left and right d-pad. This is imprecise, tedious, and inconvenient. Ideally, there would be fast-forward and rewind options at various speeds. This would not necessarily have to be double- or quadruple-speed playback, just the ability to quickly skip forward and back without having to press the same button over and over, similar to most Netflix clients and DVRs.
In general, I personally would like a less pointer-centric interface, if only because setting up the sensor bar is a pain, but I’m guessing most people don’t really care.