Tag: sparsity
-
A simple property of sparse vectors
This came up in Chapter 7 of Wainwright’s “High-dimensional Statistics”. In that Chapter we’re interested in determining how close solutions $\hat \theta$ to different flavours of the Lasso problem come to the true, $S$-sparse vector $\theta^*$. A useful notion is the set of $S$-dominant vectors (my terminology): $$ C(S) = \{x: \|x_{S^c}\|_1 \le \|x_S\|_1\},$$ i.e.…
-
Estimating the intrinsic dimensionality of data with the participation ratio
Many datasets are samples of the values of a given set of $N$ features. We can visualise these data as points in an $N$-dimensional space, with each point corresponding to one of the samples. Visualization encourages geometric characterization. A basic geometric property is dimensionality: what is the dimension of the space in which the data…