Sustained contralateral delay activity (CDA) emerges in the retention period of working memory (WM) tasks and has been commonly interpreted as an electrophysiological index of the number of items held in a discrete-capacity working memory resource. More recent findings indicate that these visual and tactile components are sensitive to various cognitive operations beyond the storage of discrete items in WM. In this Perspective, we present recent evidence from unisensory and multisensory visual and tactile WM tasks which suggest that, in addition to memory load, sensory delay activity may also be indicative of attentional and executive processes, as well as reflecting the flexible, rather than discrete, allocation of a continuous working memory resources. Together, these findings challenge the traditional model of the functional significance of the CDA as a pure measure of item load, and suggest that it may also reflect executive, attentional, and perceptual mechanisms operating in hierarchically organized WM systems.