Contextual Bandits

Usage with Contextual Multi-Armed Bandits

Eppo also supports contextual multi-armed bandits. You can read more about them in the high-level documentation. Bandit flag configuration--including setting up the flag key, status quo variation, bandit variation, and targeting rules--are configured within the Eppo application.

To leverage bandits using the JavaScript Client SDK, there are two additional steps over regular feature flags:

1. Add a bandit action logger to the SDK client instance
2. Provide the SDK the available actions for the bandit

Bandits can only be used with precomputed assignments.

await EppoSdk.precomputedInit({
    apiKey: 'YOUR_SDK_KEY',
    assignmentLogger: {
        logAssignment: (assignmentEvent) => {
            /* send to warehouse */
        }
    },
    banditLogger: {
        logBanditAction: (banditEvent) => {
            /* send to warehouse */
        }
    },
    precompute: {
        subjectKey: 'test-subject',
        subjectAttributes: { attr1: 'value1' },
        banditActions: {
            'bandit1-key': {
                'action1-key': {
                    numericAttributes: { subjectActionNumericAttr1: 12345 },
                    categoricalAttributes: { subjectActionCategoricalAttr1: 'value1' }
                },
                'action2-key': { numericAttributes: {}, categoricalAttributes: {} },
                'action3-key': { numericAttributes: {}, categoricalAttributes: {} },
                'action4-key': { numericAttributes: {}, categoricalAttributes: {} }
            }
        }
    }
});

The SDK will invoke the logBanditAction() function with an IBanditEvent object that contains the following fields:

Field	Description	Example
timestamp (string)	The time when the action is taken in UTC as an ISO string	"2024-03-22T14:26:55.000Z"
featureFlag (string)	The key of the feature flag corresponding to the bandit	"bandit-test-allocation-4"
bandit (string)	The key (unique identifier) of the bandit	"ad-bandit-1"
subject (string)	An identifier of the subject or user assigned to the experiment variation	"ed6f85019080"
subjectNumericAttributes (Attributes)	Metadata about numeric attributes of the subject. Map of the name of attributes their provided values	{"age": 30}
subjectCategoricalAttributes (Attributes)	Metadata about non-numeric attributes of the subject. Map of the name of attributes their provided values	{"loyalty_tier": "gold"}
action (string)	The action assigned by the bandit	"promo-20%-off"
actionNumericAttributes (Attributes)	Metadata about numeric attributes of the assigned action. Map of the name of attributes their provided values	{"brandAffinity": 0.2}
actionCategoricalAttributes (Attributes)	Metadata about non-numeric attributes of the assigned action. Map of the name of attributes their provided values	{"previouslyPurchased": false}
actionProbability (number)	The weight between 0 and 1 the bandit valued the assigned action	0.25
optimalityGap (number)	The difference between the score of the selected action and the highest-scored action	456
modelVersion (string)	Unique identifier for the version (iteration) of the bandit parameters used to determine the action probability	"v123"
metaData Record<string, unknown>	Any additional freeform meta data, such as the version of the SDK	{ "sdkLibVersion": "3.5.1" }

Querying the bandit for an action

To query the bandit for an action, you can use the getBanditAction() function. This function takes the following parameters:

• flagKey (string): The key of the feature flag corresponding to the bandit
• defaultValue (string): The default variation to return if the flag is not successfully evaluated

The following code queries the bandit for an action:

const banditResult = EppoSdk.getPrecomputedInstance().getBanditAction('bandit1-key', 'bandit1-default');
const variant = banditResult.variant;
const action = banditResult.action;

Subject context

The subject context contains contextual information about the subject that is independent of bandit actions. For example, the subject's age or country.

The subject context can be provided as Attributes, which will then assume anything that is number is a numeric attribute, and everything else is a categorical attribute.

You can also explicitly bucket the attribute types by providing the context as ContextAttributes. For example, you may have an attribute named priority, with possible values 0, 1, and 2 that you want to be treated categorically rather than numeric. ContextAttributes have two nested sets of attributes:

• numericAttributes (Attributes): A mapping of attribute names to their numeric values (e.g., age: 30)
• categoricalAttributes (Attributes): A mapping of attribute names to their categorical values (e.g., country)

Any non-numeric values explicitly passed in as values for numeric attributes will be ignored.

Attribute names and values are case-sensitive.

note

The subject context, passed in as the subjectAttributes parameter, is also still used for targeting rules for the feature flag, just like with non-bandit assignment methods.

Action contexts

The action context contains contextual information about each action. They can be provided as a mapping of attribute names to their contexts.

Action contexts must be provided as ContextAttributes, which have explicit bucketing into numericAttributes and categoricalAttributes.

Note that relevant action contexts are subject-action interactions. For example, there could be a "brand-affinity" model that computes brand affinities of users to brands, and scores of that model can be added to the action context to provide additional context for the bandit.

If the subject is assigned to the variation associated with the bandit, the bandit selects one of the supplied actions. All actions supplied are considered to be valid. If an action should not be available to a subject, do not include it in initialization.

Like attributes, actions and their attributes are case-sensitive.

Result

getBanditAction() returns two fields:

• variation (string): The variation that was assigned to the subject
• action (string | null): The action that was assigned to the subject by the bandit, or null if the bandit was not assigned

The variation returns the feature flag variation. This can be the bandit itself, or the "status quo" variation if the subject is not assigned to the bandit.

If we are unable to generate a variation, for example when the flag is turned off, then the provided default variation is returned. In both of those cases, the returned action will be null, and you should use the status-quo algorithm to select an action (more on this below).

When action is not null, the bandit has selected an action for the subject.

note

If no actions are provided and the flag still has an active bandit, if the bandit variation is assigned the assigned action will be null.

note

If the flag no longer has any allocations with bandits, this function will behave the same as getStringAssignment(), with the provided actions being ignored and the assigned variation being returned along with a null action.

Status quo algorithm

In order to accurately measure the performance of the bandit, we need to compare it to the status quo algorithm using an experiment. This status quo algorithm could be a complicated algorithm to that selects an action according to a different model, or a simple baseline such as selecting a fixed or random action. When you create an analysis allocation for the bandit and the returned action is null, implement the desired status quo algorithm based on the variation value.

Debugging

You may encounter a situation where a flag assignment produces a value that you did not expect. There are functions detailed here to help you understand how flags are assigned, which will allow you to take corrective action on potential configuration issues.