Programmable quantum simulators and quantum computers are opening unprecedented opportunities for exploring and exploiting the properties of highly entangled complex quantum systems. The complexity of large quantum systems is the source of computational power but also makes them difficult to control precisely or characterize accurately using measured classical data. We review protocols for probing the properties of complex many-qubit systems using measurement schemes that are practical using today's quantum platforms. In these protocols, a quantum state is repeatedly prepared and measured in a randomly chosen basis; then a classical computer processes the measurement outcomes to estimate the desired property. The randomization of the measurement procedure has distinct advantages. For example, a single data set can be used multiple times to pursue a variety of applications, and imperfections in the measurements are mapped to a simplified noise model that can more easily be mitigated. We discuss a range of cases that have already been realized in quantum devices, including Hamiltonian simulation tasks, probes of quantum chaos, measurements of non-local order parameters, and comparison of quantum states produced in distantly separated laboratories. By providing a workable method for translating a complex quantum state into a succinct classical representation that preserves a rich variety of relevant physical properties, the randomized measurement toolbox strengthens our ability to grasp and control the quantum world.