Simple Random Sampling produces samples of a specific size, where each item has the same probability of being chosen. DataFu has scalable implementations of this that will generate samples …

Sampling UDFs, including weighted sample, reservoir sampling, sampling by key, etc.

It takes a bag of n items and a sampling probability p as the inputs, and outputs a simple random sample of size exactly ceil (p*n) in a bag, with probability at least 99.99%.

Sampling UDFs, including weighted sample, reservoir sampling, sampling by key, etc. See: Description

It takes a sampling probability p as input and outputs a simple random sample of size exactly ceil (p*n) with probability at least 99.99%, where $n$ is the size of the population.

We can simply draw a number from this distribution, determine the positions by sampling without replacement, and then generate random scores for those positions.

The method of sampling is to convert the key to a hash, derive a double value from this, and then test this against a supplied probability. The double value derived from a key is uniformly distributed …

All Classes Summary: Nested | Field | Constr | Method Detail: Field | Constr | Method datafu.pig.sampling

Set Operations: set intersection, union, difference Sessions: sessionize streams of data Sampling: simple random sample with/without replacement, weighted sample, sample by keys Hashing: SHA …

datafu.pig.sampling Sampling UDFs, including weighted sample, reservoir sampling, sampling by key, etc. datafu.pig.sessions