Sequential games and optimal strategies

This article gives an overview of recent work on the theory of selection functions. We explain the intuition behind these higher type objects, and define a general notion of sequential game whose optimal strategies can be computed via a certain product of selection functions. Several instances of this game are considered in a variety of areas such as fixed point theory, topology, game theory, higher type computability and proof theory. These examples are intended to illustrate how the fundamental construction of optimal strategies based on products of selection functions permeates several research areas.


Introduction
Life is the sum of all your choices, so said Albert Camus. But what does 'choice' mean? One could say that to choose is to select one element x out of a set X of possible candidates. The set X could be the collection of all the petrol stations on the way to work, or the different types of fuel available at a particular station. Our choice x is which petrol station to use when the fuel tank empties or which types of fuel we prefer.
So, how do we make choices? What leads us to pick one particular x over all others? If we ignore the fact that in some cases we do like to pick things at random, it is fair to say that we normally decide on a few criteria before making a choice. For instance, quite often we implicitly assign a value (say, a real number R) to each element of the set, representing the cost of making that particular choice. If X is the set of fuel types at a particular petrol station, then our mapping p: X → R could be the cost of a litre of petrol of each particular type x. We then select an x ∈ X that has minimal cost or the best cost-benefit. In fact, we are only using the real numbers as an ordered set. Any ordered set (R, ≤) would be sufficient. The order does not even need to be total, meaning that for some two candidates x and x we might not be able to tell which one is better. *Author for correspondence (pbo@dcs.qmul.ac.uk).
Anyway, once we have fixed a more structured set R, our 'criteria' mapping p: X → R propagates the structure of R to the elements of X . For instance, if R is an ordered set, we immediately get an order on X by saying that a petrol station x is better than another petrol station y, if p(x) ≥ p(y). Thus, we might be interested in picking an x ∈ X , which either maximizes or minimizes the value of p(x), depending on whether p: X → R represents the quality of fuel or the cost of a litre of that particular fuel type, respectively.
Although the mapping p: X → R depends on the criteria used by a particular individual, the fact that we pick an x, which minimizes the cost is much more universal. We use the same minimization function when choosing the cheapest hotel or the cheapest pair of trousers. Any function, such as the minimization or maximization functions, that in a uniform way picks an x ∈ X when given a mapping p: X → R we shall call a selection function. These are functions of type The 'minimization' selection function, which we use when p describes the cost associated with each particular choice, is 3(p) = any x ∈ X , such that p(x) ≤ p(y) for all y ∈ X , whereas the 'maximization' selection functions, which we use when p represents the payoff obtained by making a particular choice, is d(p) = any x ∈ X , such that p(x) ≥ p(y) for all y ∈ X .
Both 3 and d have type (X → R) → X .
Note that selection functions decide which element to choose for each given 'criteria' mapping p: X → R. If the cost of the different fuel types change, we might be led to make a different choice, although we are still using the same selection function 3. Only the criteria mapping p will have changed.
Once we have the chosen element x = 3(p), we can in particular calculate the optimal cost (or payoff) we have achieved as r = p(x) = p(3(p)).
For a given p: X → R, we say that x = 3(p) is an optimal choice, whereas r = p(3(p)) is the optimal outcome. We can also define the optimal outcome directly. For instance, in the case of the petrol station, the optimal outcome is the minimum value of the mapping p: X → R. As 3 picks an x where p has least value, we have that min(p) = p(3(p)), for all p: X → R. A similar equation to the above also holds for the relation between max and the selection function d, i.e. max(p) = p (d(p)).
In general, we call a function f: (X → R) → R a quantifier. When for a given quantifier f a selection function 3 always picks x = 3(p), such that f(p) = p(x) we say that 3 is a selection function for f. In this case f can be defined from epsilon by the equation f(p) = p(3(p)).
Intuitively, both selection functions and quantifiers operate on the criteria mapping p: X → R. A quantifier f says what is the desired outcome for a given p: X → R, whereas a selection function 3 for f calculates an x ∈ X , which achieves that outcome. Note that our concept of 'quantifier' is more general than Mostowski's notion of a generalized quantifier (Mostowski 1957), which is the case when R is the set of truth-values or Booleans.
For the rest of this review, we shall explore the connection between quantifiers and selection functions, and define algebraic operations on them. We will then define a notion of sequential game, and show that the product of selection functions allows us to compute optimal strategies in such games. In §8, we show, by picking particular instantiations from different areas, that this notion of game is very general. This review mostly covers recent work of the authors, which has appeared in Escardó & Oliva (2010a,b,c), although some concepts have been simplified and generalized.
l-notation. In the following sections, we rely heavily on the use of l-notation to describe functions. For instance, the function 'two to the power of n' can be described as ln · 2 n . When this is applied to 3, for instance, it gives (ln · 2 n )(3) = 2 3 = 8. The l-notation is particularly useful when working with higher type objects such as functions 3 from X → R to X , since its argument is itself a function.
Models of higher type functions. For the first part of the paper ( § §2-5), which concerns finite games of fixed length, the reader can safely think of X and Y as ordinary finite or infinite sets, and the mappings X → Y as the set of all functions from X to Y . In §6, however, which concerns games of unbounded length, one has to be careful and work with (non-standard) models of functionals, so as to ensure that the infinitely iterated product is well-defined. The model we should bear in mind in this case is that of the continuous functionals (Normann 1999), and in some cases also the model of majorizable functionals (Bezem 1985).

The product of quantifiers
Continuing from the examples given in §1, suppose now that we could only choose which petrol station to stop at, and the owner of the petrol stations then chooses the type of fuel we must use at that particular station we have picked. As we know that the owner will try to maximize his profit and will offer us the most expensive fuel, we have to choose the petrol station with the cheapest most expensive fuel. In other words, if X is the set of petrol stations, Y is the set of different fuel types, and q: X × Y → R gives the cost of fuel y at station x, then the best outcome for us is to pay Interestingly, this is also the optimal outcome for the owner of the petrol stations, as he has no control over which station we choose. Let us use the abbreviations K R X = (X → R) → R and J R X = (X → R) → X for the types of quantifiers and selection functions, respectively.

Definition 2.1 (Binary product of quantifiers)
. Given quantifiers f: K R X and j: K R Y , we define their product (f ⊗ j): For instance, given the quantifiers inf: K R X and sup: K R Y discussed above we have that their product is where, for clarity, we indicate above the equality sign the type of the two objects. Also, given the quantifiers ∃ : K B X and ∀ : K B Y , corresponding to the existential and universal quantifiers, we have that their product concisely describes the nesting of quantifiers, i.e.
We can iterate this binary product of quantifiers and obtain an n-ary product a follows: Definition 2.2 (n-ary product of quantifiers). Given a sequence of quantifiers f i : K R X i , 0 ≤ i ≤ n, define their product inductively by iterating the binary product as Let us show how the concept of quantifiers allows us to define a very general notion of sequential games. We will also show that the product of quantifiers described above computes the optimal outcome of such games.

Finite games of fixed length
Games often simulate life. In particular, a play in a game can be viewed as a sequence of choices that lead to some outcome. Hence, we can think of a sequence of choices as a sequence of moves in some sequential game. In the case of choosing which petrol station to use, this is a simple game with one player and one round. In the case, we must also choose which fuel type to use, this is a one-player tworound game. If the owner of the station chooses the type of fuel we must use, then this is a two-player two-round game. In all cases, the aim of the game for us (the first player in the last example) is to minimize the cost (or maximize the payoff) obtained from a particular choice, and the actual move in the game is the particular choice we commit ourselves to.
In the following, we explore the concept of a quantifier introduced above and generalize these scenarios to games with a finite number of rounds.
Definition 3.1 (Finite game). Any tuple (R, (X i ) n−1 i=0 , (f i ) n−1 i=0 , q) defines an n-round sequential game, where -R is the set of possible outcomes of the game, -X i is the set of possible moves at round i, -f i : K R X i is the quantifier for round i, and -q: P n−1 i=0 X i → R is the outcome function.
A sequence x 0 , . . . , x n−1 ∈ P n−1 i=0 X i is called a play of the game. Note that, we have said nothing about the number of players in the game. Instead, the definition of the game includes a quantifier f i for each round. This function f i picks a particular outcome f i (p) = r ∈ R when given a mapping p: X i → R. Intuitively, once we know how each choice of move in X i will impact the outcome of the game, then f i tells us what outcome would be desirable.
For instance, if R = {−1, 0, 1} and f i are the minimum-value function for odd rounds and the maximum-value function for even rounds, then we have a two player game where one player is aiming for the outcome −1, whereas the other is aiming for the outcome 1. Thus, the three possible outcomes correspond to one of the players winning, or the game ending in a draw.
The example in §2 corresponds to the case when n = 2, i.e. a game with two rounds. X is the set of petrol stations and Y is the collection of different fuel types. The outcome function q: X × Y → R gives the cost of a particular fuel at a particular station. As we are choosing which station to use, and the owner of the stations chooses the fuel to be used, we have that f 0 = inf, whereas f 1 = sup.

Definition 3.2 (Strategy). A strategy for the game (R, (X
Given a partial play a = a 0 , . . . , a k−1 , any strategy defines a strategic extension of a as b a j X j = next j (a 0 , . . . , a k−1 , b a k , . . . , b a j−1 ) for k ≤ j < n.
Given a strategy (next k ) i<n , the functions next k compute which move should be played at each round k, i.e. when the game is at position a = a 0 , . . . , a k−1 the next move selected is given by a k = next k (a). The strategic extension of a partial play is simply the play obtained by following the strategy at each round.

Definition 3.3 (Optimal strategy).
A strategy (next k ) k<n is said to be optimal if for every partial play a = a 0 , . . . , a k−1 , we have a,x k n−1 )). Intuitively, a strategy is optimal if the outcome obtained by following the strategy q(a, b a k , . . . , b a n−1 ) is the best possible outcome as described by the quantifier f k , for each round k. For instance, considering again the example from §2, the optimal strategy would consist of the functions next 0 () = any x ∈ X , such that sup q x ≤ sup q x for any other x ∈ X and next 1 (x) = any y ∈ X , such that q(x, y) ≥ q(x, y ) for any other y ∈ Y , where sup q x abbreviates sup y∈Y q(x, y). In general, we will write q x for the function ly · q(x, y).

Theorem 3.4 (Optimal outcome).
For any optimal strategy (next k ) k<n let b be its strategic play. We have that We call w the optimal outcome of the game. Note that, this is unique and independent of the optimal strategy.
Proof. Let an optimal strategy (next k ) k<n be fixed. By the definition of optimal strategy (definition 3.3), we have As discussed in §2, the optimal outcome of the game between us and the owner of the petrol station is for us to choose the station with cheapest most expensive fuel, whereas the owner then picks the most expensive fuel at the station we have chosen. So the optimal outcome of the game is If, on the other hand, we could choose not only which station to use, but also which fuel type to use (and if we were interested in minimizing our cost) that is a different game where f 0 = f 1 = inf and the optimal outcome of the game is where we pick the station which has the cheapest fuel, and then pick the cheapest fuel at that chosen station.
The subgame (R, (X i ) n−1 i=k , (f i ) n−1 i=k , q a ) is like the original game except that we are starting at the position determined by the initial moves a. Notice that if k = n then q is constant, and when k = 0 this is the same as the full game. The optimal outcome of the subgame defined by a can similarly be computed as If k = n this corresponds to the subgame where there are no more moves to be made-the end of the game. Therefore, the outcome can be computed directly from the moves already made a, i.e. w a = q(a). If k < n then we have a proper subgame, and the optimal outcome of this proper subgame is Hence, the optimal outcome of round k is determined by the quantifier for round k together with the mapping lx k · w a * x k computing the optimal outcome at round k + 1 given what is played at round k. Considering again the example from §2, when choosing which petrol station x to stop at, it is enough to look at the most expensive fuel type w x at each station, because we know that this is the fuel type the owner of the stations will offer us. Therefore, the optimal outcome for us is inf(lx · w x ). The notion of optimal outcome for subgames leads to the notion of optimal move. Definition 3.6 (Optimal move and optimal play). For a partial play a = a 0 , . . . , a k−1 a move x k at round k is said to be an optimal move if it preserves the optimal outcome, i.e. if the optimal outcome w a of the subgame determined by a is the same as the optimal outcome w a * x k of the new subgame determined by a * x k . A whole play x = x 0 , . . . , x n−1 is called an optimal play if each x k is an optimal move given the partial play x 0 , . . . , x k−1 .
In other words, a move at round k is optimal if it leads to a subgame where the optimal outcome of the game is still achievable. As one would expect, if we play optimally at each round we will end-up with the optimal outcome of the game.
Lemma 3.7. If x = x 0 , . . . , x n−1 is an optimal play then i.e. the outcome of an optimal play is the optimal outcome. Proof. Since x is an optimal play we have that (for all 0 ≤ i < n) x i is an optimal move in the subgame x 0 , . . . , Moreover, the next lemma shows that plays obtained by following optimal strategies are indeed optimal.
Lemma 3.8 (Optimal strategies and optimal plays). For any optimal strategy its strategic play is an optimal play. Proof. Assume (next k ) k<n is an optimal strategy, and let b = b 0 , . . . , b n−1 be the strategic play. We must show, for 0 ≤ k < n − 1, that w a = w a * b k , where a = b 0 , . . . , b k−1 . By (i) the definition of optimal outcome and (ii) the fact that (next k ) k<n is an optimal strategy we have Note that optimal strategies do not exist in general, since optimal outcomes are not necessarily achievable. For instance, consider the game with a single round having quantifier f 0 : and assume the outcome function is q(x) = 1. The optimal outcome defined by f 0 is 0, whereas the only achievable outcome is 1. Nevertheless, in §5, we will show that optimal strategies always exist when the quantifiers that define the game are attainable, i.e. have corresponding selection functions. Moreover, a suitably defined product of selection functions calculates optimal strategies.

Games with multiple optimal outcomes
Before we proceed to show how a product of selection functions computes optimal strategies, let us first generalize the notion of game introduced in §3. For that, let us consider the space Similarly to before, we say that a multi-valued quantifier f: S R X is attainable if there exists a selection function 3: J R X , such that for all p: X → R. Intuitively, f describes a set of optimal outcomes for any given criteria mapping, and the selection function 3 for f picks an x ∈ X , which achieves one of the optimal outcomes.

Definition 4.1 (Finite games with multiple optimal outcomes
). An n-round sequential game with multiple optimal outcomes is a tuple (R, -R is the set of possible outcomes of the game, -X i is the set of possible moves at round i, -f i : S R X i is the multi-valued quantifier for round i, and -q: A sequence x 0 , . . . , x n−1 ∈ P n−1 i=0 X i is called a play of the game.
Just as in §3, every partial play a: k−1 i=0 X i defines a subgame. In games with multiple optimal outcomes, however, we cannot define 'the' optimal outcome as we did in §3, simply because there might not be a single optimal outcome for the whole game. Moreover, there seems to be no way of defining a product of multi-valued quantifiers, as we did for the single-valued quantifiers. Nevertheless, we can still define a notion of an optimal strategy, slightly generalizing definition 3.3.

Definition 4.2 (Optimal strategy).
Recall that a partial play a = a 0 , . . . , a k−1 and a strategy ( for k ≤ j < n. A strategy (next k ) k<n is said to be optimal if for every partial play a = a 0 , . . . , a k−1 , we have Compare this with definition 3.3, where we have equality (=) instead of ∈.
In §3, we said that a strategy was optimal if the outcome of an strategic play q(a, b a k , . . . , b a n−1 ) was precisely the optimal outcome according to the quantifier f k . By moving to multi-valued quantifiers, what we must say instead is that a strategy is optimal if the outcome of a strategic play is among the possible optimal outcomes for each of the multi-valued quantifier f k . It is clear that when multivalued quantifiers are in fact single-valued the two notions of optimal strategy (definitions 3.3 and 4.2) coincide.

Calculating optimal strategies
We have explained at the end of §3, why optimal outcomes are not always achievable. Note also that even when an optimal outcome is achievable, optimal strategies are not necessarily unique. There might be several choices which are all equally optimal, if all lead to an optimal outcome of the game. Nevertheless, what we show next is that if the (multi-valued) quantifiers f k for each round k have associated selection functions 3 k , then we can always compute optimal strategies (in the more general sense of definition 4.2) in the game defined by such f k 's. Intuitively, having a selection function 3 is a stronger requirement than simply having a quantifier f, because every selection function 3 defines a (single-valued) quantifier as f(p) := p(3(p)).
Recall that J R X abbreviates (X → R) → X .
Definition 5.1 (From J R X to K R X). Denote by (·): J R X → K R X the following mapping from selection functions to quantifiers 3(p) := p(3(p)).
(5.1) Going back to the example from §2, we are in a good position because the quantifiers inf and sup in this case have selection functions 3 and d, respectively (as defined in §1). It is for that reason that we can find a play which is optimal and hence leads to the optimal outcome inf x∈X sup y∈Y q(x, y).
But how do we compute the optimal play x, y that leads to this optimal outcome? The solution is to turn the game upside down. Instead of trying to find the correct x first, we instead look for the best choice of y ∈ Y for any given x ∈ X . That can be done very easily using the selection function d for the quantifier max, i.e. y x := d(ly · q(x, y)).
Hence, for each of our choices of petrol station x ∈ X , we know what the optimal move y x ∈ Y of the second player is. Once we have this mapping x → y x from our choice of station x to the fuel type y x ∈ Y the owner of the petrol station will offer us, we can choose the petrol station with cheapest most expensive fuel. For that we use the selection function for the minimization quantifier, i.e. a := 3(lx · q(x, y x )).
Finally, once we have computed our best choice of petrol station a as above, we can then go back to y x and compute the optimal move for the second player as y = y a (where a is now the optimal move for player one). Such construction works in general, as we have only used the fact that 3 and d are selection functions for the quantifiers min and max, respectively.

Definition 5.2 (Binary product of selection functions). Given selection functions
where a = 3(lx · q(x, y x )) and y x = d(ly · q(x, y)). We can then equivalently define the product above using function composition as First let us show that the product of selection functions corresponds to the product of single-valued quantifiers defined in §2.
Lemma 5.3. For any given 3: J R X and d: Proof. Simply unfolding definitions we have From this we can conclude that, just like the product of quantifiers (definition 2.1) computes the optimal outcome of a single-optimal-outcome game, the product of selection functions (definition 5.2) computes an optimal play in such games.

Theorem 5.4 (Optimal plays for two-round games). Let a two-round sequential
If the quantifiers f 0 , f 1 have selection functions 3 0 , 3 1 then an optimal play in the game can be computed as x, y = (3 0 ⊗ 3 1 )(q).
In the general case of multi-valued outcome functions, we have: Theorem 5.5 (Optimal strategies for two-round games-multi-valued f i 's). Let a two-round sequential game (R, (X i ) 1 i=0 , (f i ) 1 i=0 , q) be given, where f 0 : S R X 0 and f 1 : S R X 1 are multi-valued quantifiers. If f 0 , f 1 have selection functions 3 0 , 3 1 then an optimal strategy in the game can be computed as next 0 () = p 0 ((3 0 ⊗ 3 1 )(q)) and next 1 (x) = 3 1 (ly · q x (y)), where p 0 denotes the first projection, i.e. p 0 (x, y) = x.
Just as we did with the binary product of quantifiers (definition 2.2) we can iterate the binary product of selection function to obtain an n-ary product as follows: Definition 5.6 (n-ary product of selection functions). Given a sequence of selection functions 3 i : J R X i define their product by simply iterating the binary product as where (⊗ n i=n 3 i ) = 3 n . The following theorem describes how this product of selection functions can be used to compute optimal strategies. Theorem 5.7 (Main theorem for finite games of fixed length). Let an n-round sequential game (R, (X i ) n−1 i=0 , (f i ) n−1 i=0 , q) be given (in the more general sense of definition 4.1). If the multi-valued quantifiers f i : S R X are attainable, i.e. have associated selection functions 3 i , then an optimal strategy for the game can be computed as Proof. This is a particular case of the more general theorem 6.5, which we prove in §6.
We call the optimal strategy computed from 3 the 3-strategy, and its strategic play we call the 3-play. As a consequence of the above theorem we obtain a general way of finding a solution a and p to the following simple system of equations and membership. We will discuss in §7 a few examples of how such a system of equations appears in practice.
Corollary 5.8. Let an n-round sequential game (R, (X i ) n−1 i=0 , (f i ) n−1 i=0 , q) be given. If the multi-valued quantifiers f i : S R X are attainable then there are functions p k : X k → R, such that a k = 3 k (p k ) and q(a) ∈ f k (p k ), Proof. This is a particular case of the more general corollary 6.6 which we prove in §6.
In particular, note that if f k are all single-valued then p j (a j ) = p k (a k ), for all 0 ≤ j, k < n, as The intuitive reading of corollary 5.8 is that in any sequential game (with multivalued outcome functions) it is always possible to turn the global outcome function q: P n−1 i=0 X i → R into a family of local outcome functions p k : X k → R, so that -the 3-play a can be computed locally from the selection functions 3 k , and -the outcome of the 3-play is in the intersection of 'desired' outcomes for all rounds k.

Finite games of unbounded length
The sort of games considered so far terminate after a fixed number of rounds n.
We will show next that it is also possible to compute optimal strategies in games with an unbounded (but finite) number of rounds. These are games where the number of rounds until the outcome is reached depends on how the game develops. Because the number of rounds in the game can be arbitrarily large, we must include in the definition of the game a set of possible moves X i and an outcome quantifier f i for all natural numbers i ∈ N. However, we do not want to consider games that go on indefinitely. One way to ensure that every play in the game is finite is to assume that the value of the outcome function q is determined after a finite (but unbounded) number of rounds, i.e.
∀a∃n∀b(∀i ≤ n(a(i) For any 'infinite' play a there exists a point n where the value of the outcome q(a) is fixed, so that any other extension b of the play would lead to the same outcome. This holds, for instance, when the set of outcomes R is discrete 1 and the outcome function q is continuous.

Definition 6.1 (Finite but unbounded game). A finite but unbounded sequential game is a tuple (R, (X
-R is the discrete set of possible outcomes of the game, -X i is the set of possible moves at round i ∈ N, -f i : S R X i is the multi-valued quantifier for round i ∈ N, and -q: P ∞ i=0 X i → R is the continuous outcome function. A sequence a ∈ P ∞ i=0 X i is called a play of the game. The relevant part of a play a is the finite initial segment a(0), . . . , a(n) ∈ P n i=0 X i which determines the value of q(a).
As in §3, a strategy for an unbounded game (R, ( is a family of mappings next k : k−1 i=0 X i → X k , for each k ∈ N. Given a partial play a = a 0 , . . . , a k−1 , any strategy defines a strategic extension of a as b a (j) = next j (a 0 , . . . , a k−1 , b a (k), . . . , b a (j − 1)) for j ≥ k.

Definition 6.2 (Optimal strategy).
A strategy (next k ) k∈N is said to be optimal if for all k ∈ N and for every partial play a = a 0 , . . . , a k−1 , we have q(a * b a ) ∈ f k (lx k · q(a * x k * b a,x k )).
Having defined the unbounded game and its generalized notion of optimal strategy, we now show how the infinite iteration of the binary product of selection functions computes optimal strategies in games where the quantifiers f i are attainable. Definition 6.3 (Unbounded product of selection functions). Given an infinite sequence of selection functions 3 i : J R X i define their unbounded product by simply iterating the binary product as where, for clarity, the type of the final product is shown above the equality sign.
It is perhaps surprising that such product functional is in fact not only well-defined in the model of continuous functionals (Scarpellini 1971) but also computable and part of the standard installation of Haskell (see §7g for more details). This is in stark contrast with the iterated product of quantifiers which is not well-defined even in the model of continuous functionals.
We will show next that the above product of selection functions permits us to compute optimal strategies for finite but unbounded games with attainable quantifiers. First, let us prove the following useful lemma. For all j ∈ N the following holds: where (a)[j] is the initial segment of a of length j, i.e. a(0), . . . , a(j − 1).
Proof. If j = 0 this follows by definition. Assuming this holds for j we wish to show it for j + 1. We have * (a)[j] * y )) = a(j). Therefore, we have that a = (a)[j + 1] * ( ∞ i=j+k+1 3 i )(q x * (a)[j+1] ). As with the finite games of fixed length (theorem 5.7) we now show how the unbounded product of selection functions (definition 6.3) computes optimal strategies in games of unbounded length.

q) be given. If the quantifiers f i have selection functions 3 i then an optimal strategy in the game can be computed as
Proof. Fix an arbitrary partial play By lemma 6.4 we also have that By the fact that 3 k is a selection function for f k we get The result follows since q(x, b x (k) * b x,b x (k) ) = q(x * b x ), i.e. starting a strategic play from x or an extension of that with the next strategic play x, b x (k) leads to the same final play.
As a corollary, we obtain that the following system of equations and membership can be solved via the product of selection functions. This generalizes corollary 5.8. Corollary 6.6. Let a finite but unbounded sequential game (R, (X i ) ∞ i=0 , (f i ) ∞ i=0 , q) be given. If the quantifiers f k have associated selection functions 3 k then there are functions p k : X k → R, such that for k ∈ N, where a is the 3-strategic play from the proof of theorem 6.5, i.e.
Moreover, since next k is an optimal strategy we have, for all k ∈ N that

(a) Variant of finite games of unbounded length
The generalization of finite games to unbounded games considered above relies on the set of outcomes R being discrete (and q being continuous). What if R is not discrete, e.g. R = R? Another way to ensure that every play in the game is finite is to assume that each outcome r ∈ R determines the length of the play to be considered, via a fixed mapping l: R → N, which we will call the 'clock function'. Definition 6.7 (Unbounded game with explicit control). A finite but unbounded sequential game with explicit control is a tuple (R, ( -R is the set of possible outcomes of the game, -X i is the set of possible moves at round i, -f i : S R X i is the multi-valued quantifier for round i, -q: P ∞ i=0 X i → R is the outcome function, and -l: R → N is the clock function. 2 A sequence a ∈ P ∞ i=0 X i is called a play of the game. The relevant part of a play a is the finite initial segment a 0 , . . . , a n ∈ P n−1 i=0 X i of length n = l(q(a)). Therefore, only finite initial segments of infinite plays are to be considered, although the non-relevant extension may be crucial to deciding the outcome of the game (a different non-relevant extension might lead to a different outcome). In fact, the length of the relevant part of the infinite play is only determined once we have fixed the infinite play a. As we are only interested in finite games, we assume that for each set of moves X i there exists a canonical element c i ∈ X i , and that c k ∈ P ∞ i=k X i is the infinite sequence of canonical elements from X k onwards. In this way, after each finite play x we check whether the canonical extension of x is such that its relevant part is an initial segment of x. If so, we have already played past the relevant part of the game and we can stop, since we only require that the first n = l(q(a)) moves be optimal. All other (non-relevant) moves might not necessarily be optimal.
Although this sort of unbounded finite game seems less natural than the one introduced in definition 6.1, we show in §7 that it does appear naturally in proof theory, from computational interpretations of ineffective principles (see §7f ).
We now show how to compute plays which have an optimal relevant part. First, we must adapt the notion of optimal strategy to take into account the explicit control function l: R → N. Definition 6.8 (Optimal strategy for variant of unbounded game). A strategy (next k ) k∈N is said to be optimal if for every partial play a = a 0 , . . . , a k−1 , such that k ≤ l(q(a * b a )) we have *  x k  *  b a,x k )).
In particular, notice that if next k is an optimal strategy in the above sense, then for every partial play a = a 0 , . . . , a k−1 we have for all k ≤ j ≤ l(q(a * b a )), where b is the strategic play of length j − k, starting from a. This is obtained by a simple iteration, always taking a to be longer and longer along the strategic play, until the condition k ≤ l(q(a * b a )) no longer holds. As in § §5 and 6, we now define a slightly different iterated product of selection functions that computes such optimal strategies for unbounded games with explicit control.

Definition 6.9 (Unbounded product of selection functions with explicit control).
Let l(·): R → N be fixed. Given an infinite sequence of selection functions 3 i : J R X i and an outcome function q: P ∞ i=0 X i → R, we define the unbounded product of (3 i ) i∈N by simply iterating the binary product of selection functions as The following theorem shows how (as in theorems 5.7 and 6.5) the product of selection functions defined above computes optimal strategies for unbounded games with explicit control.

Theorem 6.10 (Main theorem for unbounded games with explicit control).
Let an unbounded sequential game with explicit control (R, , q, l) be given. If the quantifiers f i have associated selection functions 3 i then an optimal strategy in the game can be computed as Proof. Let x = x 0 , . . . , x k−1 and assume l(q(x * b x )) ≥ k. First, note that this implies l(q(x * c)) ≥ k, as otherwise, by the definition of the strategy next k we would have that b x = c, and get a contradiction. Hence, by the definition of the strategic move at round k we have By a variant of lemma 6.4 to the explicitly controlled product, each component of the product of selection functions above corresponds to a strategic move from the initial partial play x, By the fact that 3 k is a selection function for f k we get As a corollary, we obtain that the following set of equations can be solved via the product of selection functions. The following is another possible generalization of corollary 5.8. Since we no longer require R to be discrete, but assume a clock function instead, corollary 6.11 seems to be orthogonal to corollary 6.6.

Corollary 6.11. Let an unbounded game
, q, l) with explicit control be given. If the quantifiers f i have associated selection functions 3 i then there are functions p i : for 0 ≤ i ≤ l(q(a)), where a is the 3-strategic play arising from the optimal strategy described in theorem 6.10.
Moreover, since next k is an optimal strategy we have for all k ≤ l(q(a)).

Selection functions everywhere
In this section, we show how these natural notions of finite (definitions 3.1 and 4.1) and unbounded (definitions 6.1 and 6.7) sequential games, and the corresponding computation of optimal strategies, has appeared in several different areas. We start each subsection by defining a particular instance of the game, fixing the the set of outcomes R, the sets of moves X i , the (multi-valued) quantifiers f i and the outcome function q: P n−1 i=0 X i → R. After each instantiation a discussion will follow showing how the derived optimal strategy, in case f i are attainable, corresponds to a known concept in a particular field of research. In some cases we will assume the reader is familiar with particular notions of each field, as it would be difficult to give all background necessary for each particular application discussed.
-Outcome function. For the outcome function take a Boolean formula on n variables, i.e. q: B n → B.
First, note that the optimal outcome w = n−1 i=0 f i (q) = ∃x 1 ∈ X 1 , . . . ∃x n−1 ∈ X n−1 q(x 0 , . . . , x n−1 ) calculates whether the formula q is satisfiable or not, whereas an optimal play a returns a satisfying assignment which makes q true in case the formula is satisfiable. Note also that are selection functions for f i : (X i → B) → B, i.e. given any p: By theorem 5.7 the product of selection functions computes an optimal play, i.e. a satisfying assignment in case the formula is satisfiable. Observing how the product of selection functions operates in this particular case, we note that the computation of the satisfying assignment is done backtracking. Each individual selection function 3 i performs an exhaustive search on the space B in order to find a witness for p: X i → B, when put together, the product of selection function is then able to perform an exhaustive search on the product space B n .
We note, however, that a search only happens when the formula q queries a particular variable x i . Therefore, if the value of q is already decided given a few particular values the backtracking terminates that search branch and tries a different branch. In other words, in this particular case the products of selection functions perform backtracking with 'automatic' pruning.

(b) Fixed point theory: Bekič's lemma
Consider the following instance of the finite game with multi-valued quantifiers (definition 4.1).
-Outcomes and moves. The sets of moves X i are fixed sets each endowed with a fixed point operator We are assuming now to be working in some particular model or 'domain' where such fixed point operators exist. The set of outcomes will be the product space R ≡ P n−1 i=0 X i . -Quantifiers. The multi-valued quantifiers f i : (X i → R) → 2 R are given as is a given fixed self-map on the product space.
First we note that 3 i (p X i →R ) = fix i (p i • p) are selection functions for the multivalued quantifiers f i : (X i → R) → 2 R , i.e. given any p: X i → R, we have By theorem 5.7 we can compute an optimal strategy in this game as This optimal strategy induces an strategic play a which, by corollary 5.8, satisfies for the given family of functions Unwinding the definitions of f i and 3 i we, in fact, have and (q(a)) k = fix k (p k • p k ).
Hence, a k = (q(a)) k , for all 0 ≤ k < n, which is equivalent to saying that q(a) = a. That is, the tuple a is a fixed point for the mapping q: P n−1 i=0 X i → P n−1 i=0 X i . What we have shown is that in this particular instance of the game, the optimal strategy induces a strategic play which computes a fixed point for a mapping on the product space given a family of fixed point operators for each of the individual spaces X i . Moreover, the whole construction shows that if each space X i has a fixed point operator, then the product space P n−1 i=0 X i must also have a fixed point operator. That is the essential content of Bekič's lemma (Bekič 1984), a celebrated result in fixed point theory, and frequently used in domain theory.

(c) Game theory: Nash equilibrium
Consider now another instance of the finite game with multi-valued quantifiers (definition 4.1) as follows: -Outcomes and moves. Fix sets X i as the sets of moves, which for simplicity we assume are finite. For the set of outcomes take n-tuples of reals, i.e. R ≡ R n . -Quantifiers. The multi-valued quantifiers f i : (X i → R) → 2 R are given as where p i : R → X i is the standard ith projection mapping. That is, f i returns all the elements in the co-domain of p which have maximal value at coordinate i.
-Outcome function. The outcome function q: P n−1 i=0 X i → R n is a given mapping from plays to n-tuples of real numbers.
First, since all quantifiers are different, we can think of this as an n-player game. Moreover, the outcome function can be thought of as calculating the payoff each of the n players gets at the end of the game. The quantifiers say that all players are trying to maximize their own payoffs, without paying attention to what impact this might have on other players. Note also that are selection functions for the multi-valued quantifiers f i : (X i → R n ) → 2 R n , i.e. given any p: X i → R n we have p(argmax(p i • p)) ∈ f i (p).
By theorem 5.7, we can compute an optimal strategy in this game as a,x k n−1 )), which, unwinding the definition of f k gives p k (q(a, b a k , . . . , b a n−1 )) = max This is saying that the move b a k suggested by the strategy leads to an outcome with maximal payoff for player k. It follows that the strategy next k is in Nash equilibrium, as no player has any incentive to unilaterally change their strategy for round k. In this example, the construction of optimal strategies via products of selection functions corresponds to backward induction, a technique used in Game Theory (Nisan et al. 2007) to compute Nash equilibria in sequential games.

(d) Higher type computability: searchable sets
Consider now the following instance of the unbounded game (definition 6.1), which generalizes the finite game of §7a.
-Outcomes and moves. Let R be the domain of Boolean values B and each X i be subsets S i of a fixed domain A. -Quantifiers. Let the quantifiers f i : (S i → B) → B be the existential quantifiers over the subsets S i ⊆ A. -Outcome function. For the outcome function we will take an arbitrary predicate on the product space of the S i 's, i.e. q: P n−1 i=0 S i → B.
First, we note that the assumption that f i are attainable correspond in this case to the sets S i being searchable, as defined in Escardó (2008).
It turns out that any searchable set (of total elements) is topologically compact, and, mimicking the Tychonoff theorem from topology, it was shown in Escardó (2008) that searchable sets are closed under countable products. For that one must essentially show that the game above defined has an optimal strategy, and that the optimal play witnesses q whenever q is a non-empty predicate.
In fact, the construction used in Escardó (2008) is a particular case of the iterated product of selection function (definition 6.3), and it was precisely this observation that led us to look at this construction in more details.

(e) Proof theory: arithmetic
In the next examples we show how the finite product of selection functions gives a computational interpretation to arithmetic, whereas the unbounded product interprets full classical analysis. Consider the principle of bounded collection for existential formulas, i.e.
This boils down to producing n, b, p given f , g, 3 n such that for some m ≤ b we have n = fb, m = 3 n p and gbm = p(3 n p), assuming f is bounded by t. We show that this can be solved using the optimal strategy of the following finite game. Let f , g and (3 n ) n≤t be given.
-Outcomes and moves. The sets of moves X i and the set of outcomes R is taken to be the natural numbers N.
Given that the quantifiers are attainable, we can compute the optimal strategy of this game, and its respective strategic play By corollary 5.8 we have that there are functions p n : N → N such that a n = 3 n (p n ) and q(a) = f n (p n ) = p n (3 n (p n )), for 0 ≤ n ≤ t. Take b = max a and n = fb (because f is bounded by t we have n ≤ t) and p = p n and m = a n . We show that this solves the set of equations (7.1).
By definition n = fb and m = a n = 3 n p n = 3 n p. By the definition of q we also have p n (3 n (p n )) = q(a) = g(max a)(a f (max a) ) = gba fb = gba n = gbm.
This is essentially the solution given in Oliva (2006) for the dialectica interpretation of the infinite pigeon-hole principle, but here it is given an intuitive game-theoretic explanation.
(f ) Proof theory: analysis The product of selection functions also appears in proof theory, in the form of bar recursion (Spector 1962). More precisely, in order to extend Gödel's consistency proof (Gödel 1958) from arithmetic to analysis, Spector arrived at the following system of equations: Given a family of (selection) functions 3 i : J Y X and functions r: X u → Y and u: X u → N find p: X → Y and a: X u , such that We show now that this system of equations can be solved from the optimal strategy of the following unbounded game with explicit control (definition 6.7): -Outcomes and moves. Let R = Y × N and X i = X .
-Quantifiers. The quantifiers f i : (X → R) → 2 R are defined from the 3 i 's as -Outcome function. The outcome function q: P n−1 i=0 X i → R is q(a) = (r(a), u(a)), where r: X u → Y and u: X u → N are as given in Spector's equations.
-Clock function. Let l: R → N be the second projection, i.e. l(a, n) = n.
By corollary 6.11, we have that for the strategic play a: X u there are p i : X → Y × N, such that a(i) q(a) ∈ f i (p i ), for 0 ≤ i ≤ l(q(a)), where a is the 3-strategic play. Unwinding the definitions of f, q and l, and taking p * i (x) = p 0 (p i (x)), we have a(i) X = 3 i (p * i ) and r(a) Y = p * i (3 i (p * i )), for 0 ≤ i ≤ u(a), which solves Spector's equations taking i = u(a) and p = p * i . In other words, the notion required to extend Gödel's consistency proof from arithmetic to analysis was the computation of optimal strategies for unbounded games with explicit control.
We have shown that in fact Spector's bar recursion is primitive recursive equivalent to the product of selection functions from definition 6.9, and that a different form of bar recursion (Berardi et al. 1998;Berger & Oliva 2005, 2006 is primitive recursive equivalent to the product of selection functions from definition 6.3. The second form of bar recursion arose from a game computational interpretation of classical logic (Coquand 1995), and it would be interesting to investigate in further detail how our notion of games corresponds to the one used to interpret classical logic. See also Aczel (2001) for more connections between strong monads and classical logic.

(g) Functional programming
We have also shown in Escardó & Oliva (2010c) that the construction J R over any cartesian closed category gives rise to a strong monad, with the monad morphism (·): J R → K R into the well-known continuation monad K R (Griffin 1990). This monad morphism assigns the quantifier3 ∈ K R A defined by equation (5.1) to a given selection function 3 ∈ J R A. Moreover, the case n = 2 of the product of selection functions turns out to be simply the canonical map that makes any strong monad into a monoidal monad. Monads are widely used in programming language semantics as a way to interpret side-effects (Moggi 1991) and in functional programming, particularly in the language Haskell, more generally as a way of structuring programs. As it turns out-see Escardó & Oliva (2010d)-the ability to iterate this binary product into an infinite product is already built into the Haskell language, once the selection monad is defined. All constructions described here can be easily implemented in functional programming, as done in Escardó & Oliva (2010d). In fact, experimental results show that the calculation of optimal strategies using the iterated product of selection functions is computationally very efficient with respect to both time and space.

Games in normal and extensive form
The games discussed in this paper are presented in what is known as normal form in the literature (von Neumann & Morgenstern 1944). This means that one is given an outcome function q that associates outcomes to plays (sequences of moves). For games presented in extensive form one organizes the set of all possible plays as a decision tree, as also discussed by von Neumann & Morgenstern (1944), who show that the two presentations are mathematically equivalent. For each node of the tree there is a set of edges labelled by the moves that can follow that node, leading to subtrees. Each leaf of the tree is labelled by an outcome. Given a sequence of moves, one follows the tree starting from the root until a leaf is obtained, from which one reads off the outcome of the play. The presentation of games in extensive form allows one to see that the n-ary product of selection functions (definition 5.6) corresponds to backward induction, as discussed in §7c.
For games of fixed, finite length n, all leafs are at level n of the tree. For a game of unbounded, finite length, leafs can occur at any level of the tree, but it is required that all paths eventually lead to a leaf. Trees which enjoy this property are called well founded. The well-foundedness condition for games in extensive form is equivalent to our continuity condition (6.1) for games in normal form.
Note that in the finite case, the optimal play (⊗ i<n 3 i )(q) of the game q can be seen as defined by induction on n, but this does not make sense for the infinite product of selection functions. In both the finite and infinite case, for q continuous, the outcome (⊗ i 3 i )(q) can be seen as defined by structural induction on the wellfounded tree that defines q in extensive form (see Escardó & Oliva 2010c for details). Thus, curiously, in the finite case, the same construction of (⊗ i<n 3 i )(q) can be seen at the same time as a definition by induction on n and as a definition by induction on the extensive form of q.
The above discussion shows that bar recursion, mentioned in §7f , amounts to recursion on well-founded trees, where the trees are indirectly presented as continuous functions q: n X n → R.