fairpy.metric

fairpy.metric.binary_dp(y_pred: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], s: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], abs: bool = True) → float

Demographic parity for binary sensitive attribute and binary classification

Parameters

y_pred1d array-like

Predicted labels, as returned by a classifier.

s1d array-like

Sensitive attributes.

absbool, default=True

If True, return the absolute value of dp. Otherwise, treat sensitive attribute with value 0 as the privileged group

Returns

dpfloat

Return the gap in demographic parity between two sensitive groups. The ideal performance for fairness is 0.

Examples

fairpy.metric.binary_eop(y_true: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], y_pred: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], s: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], abs: bool = True, labels: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]] | None = None) → float

Equal Opportunity for binary sensitive attribute and binary classification

Parameters

y_true1d array-like

Ground truth (correct) labels.

y_pred1d array-like

Predicted labels, as returned by a classifier.

s1d array-like

Sensitive attributes.

absbool, default=True

If True, return the absolute value of dp. Otherwise, treat sensitive attribute with value 0 as the privileged group

labelsarray-like of shape (n_classes), default=None

List of labels to index the classes.

Returns

eopfloat

Return the gap in true positive rates between two sensitive groups. The ideal performance for fairness is 0.

Examples

fairpy.metric.dcg(ranking: List[int], rel_vec: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], k: int = 0) → Tuple[float, float]

Discounted cumulative gain (dcg) and normalized discounted cumulative gain (ndcg)

fairpy.metric.xAUC(y_true: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], y_pred: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], s: Sequence[Sequence[Sequence[Sequence[Sequence[Any]]]]] | _SupportsArray[dtype] | Sequence[_SupportsArray[dtype]] | Sequence[Sequence[_SupportsArray[dtype]]] | Sequence[Sequence[Sequence[_SupportsArray[dtype]]]] | Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype]]]]] | bool | int | float | complex | str | bytes | Sequence[bool | int | float | complex | str | bytes] | Sequence[Sequence[bool | int | float | complex | str | bytes]] | Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]] | Sequence[Sequence[Sequence[Sequence[bool | int | float | complex | str | bytes]]]], abs: bool = False) → float

The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

Parameters

y_true1d array-like

Ground truth (correct) labels.

y_pred1d array-like

Predicted labels, as returned by a classifier.

s1d array-like

Sensitive attributes.

absbool, default=True

If True, return the absolute value of dp. Otherwise, treat sensitive attribute with value 0 as the privileged group

Returns

xAUCfloat

Disparity in the probabilities of ranking a positive example from one group above a negative one from another group. The best performance is 0.

Examples

from fairpy.metric import xAUC

y_true = [0, 0, 1, 0, 1, 1, 0, 1] y_pred = [0.1, 0.2, 0.3, 0.8, 0.9, 1.5, 0, 1] s = [“male”, “female”, “male”, “female”, “male”, “male”, “male”, “female”]

xauc = xAUC(y_true, y_pred, s)