Elliptic Subcovers of a Curve of Genus 2 I. The Isogeny Defect

Ernst Kani

1 Introduction

Let C be a curve of genus 2 over an arbitrary field K. An elliptic subcover is a finite morphism f : C → E to an elliptic curve E/K which does not factor over a non-trivial isogeny of E. If f′ : C → E′ is another elliptic subcover, then f′ is said to be equivalent to f if there is an isomorphism φ : E →^{∼} E′ such that f′ = φ ∘ f.

The purpose of this paper is to study the set E(C) of all equivalence classes of elliptic subcovers of a given genus 2 curve C. This is equivalent to studying the set of maximal elliptic subfields of the function field F = κ(C) of C. It is well-known that E(C) has either 0 or 2 or infinitely elements; this fact (for K = ℂ) was first noticed by Picard and Bolza; cf. Krazer[12], pp. 487-489.

Here we want to study the case that C has infinitely many elliptic subcovers. This happens precisely when the Jacobian J_{C} of C is K-isogenous to E × E, for some elliptic curve E/K.

A first step towards the problem of understanding E(C) was taken in [4] in the case that K = \overline{K} is algebraically closed. In that paper it was explained that a certain positive quadratic form q_{C}, called the refined Humbert invariant in [8], gives a (theoretical) description of E(C), for it establishes a bijection between E(C) and the set of primitive solutions x of q(x) = n^{2}, for n ∈ ℕ; cf. [4], Theorem 4.5. Here we extend this result to an arbitrary base field; cf. Theorem 20. In particular, we have

Theorem 1 The curve C/K has an elliptic subcover of degree n if and only if the refined Humbert invariant q_{C} primitively represents n^{2}.

However, in order to be able to apply this theorem to concrete situations, it is necessary to be able to compute the refined Humbert invariant q_{C} explicitly. The main objective of this paper is to present a method for computing q_{C}.

The method presented here depends on the knowledge of a “presentation (E, E′, ψ) of degree N” of C which arises from a given elliptic subcover f : C → E of degree N. This triple consists of E, another (isogenous) elliptic curve E′/K and an isomorphism ψ : E_{1}[N] → E_{2}[N] which is an anti-isometry with respect to the e_{N}-pairings; cf. Section 2. Attached to ψ is an important invariant called the isogeny defect m_{ψ} which is defined as follows:

m_{ψ} := min{m ≥ 1 : mψ = h_{|E[N]}, for some h ∈ Hom(E, E′)}.

The results are most complete (and easiest to state) when rank(Hom(E, E′)) = 1. In this case we have:

Theorem 2 Suppose that C/K is a curve which has a presentation (E, E′, ψ) of degree N with char(K) ∤ N, and let m = m_{ψ} be its isogeny defect. If Hom(E, E′) = ℤh, where d := deg(h) ≥ 1, then the refined Humbert invariant q_{C} is a positive binary quadratic form which has the following properties:

(i) disc(q_{C}) = −16m^{2}d, and m|N with (N/m, d) = 1.

(ii) q_{C} primitively represents N^{2};

(iii) q_{C}(X, Y) ≡ 0, 1 (mod 4), for all X, Y ∈ ℤ;

(iv) q_{C}(X, Y) ≠ 1, for all X, Y ∈ ℤ.

This theorem is proved at the end of Section 5, using the important Index Formula of Theorem 27 which is proved in Section 6.

In the second part[10] of this paper, Theorem 2 is made more precise by giving an explicit formula for m_{ψ} and for q_{C} in terms of an N-presentation (E, E, ψ) of C/K. In addition, we use this to prove that every binary quadratic form q satisfying properties (i)–(iv) is equivalent to) the refined Humbert invariant q_{C} of some curve C/K, when K is algebraically closed.

In the general case when r = Hom(E, E′) ≥ 1, there is a result which is similar to Theorem 2. To state it, let q_{E,E′} denote the degree quadratic form of Hom(E, E′) which is defined by q_{E,E′}(h) = deg(h), for h ∈ Hom(E, E′), and let d_{E,E′} denote its determinant.

Theorem 3 If C/K has a presentation (E, E′, ψ) of degree N with char(K) ∤ N and isogeny defect m = m_{ψ}, and if r = rank(Hom(E, E′)) ≥ 1, then the refined Humbert invariant q_{C} is a positive definite quadratic form of rank r + 1 which satisfies (the analogues of) properties (ii)-(iv) of Theorem 2, as well as the following property:

(i′) det(q_{C}) = 2^{2r+1}m^{2}d_{E,E′}.

It is interesting to note that (the proof of) Theorem 3 allows us to prove the following (partial) characterization of Jacobians which are isomorphic to a product E ×E′. Such Jacobians were studied in [8], [7] and also indirectly in [6]. One direction of this characterization was proven by Diem and Frey[1].

Theorem 4 If (E, E′, ψ) is a presentation of degree N of a curve C/K of genus 2 with char(K) ∤ N, then m_{ψ} = 1 if and only if J_{C} ≃ E × E′.

Corollary 5 If C/K has an elliptic subcover f : C → E with char(K) ∤ deg(f), and if the Picard number ρ = rk(NS(J_{C})) ≥ 3, and if det(q_{C})/2^{2ρ−3} is squarefree then J_{C} ≃ E × E′, for some elliptic curve E′/K.

Acknowledgment. This research was supported by a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (NSERC).

2 Elliptic subcovers

Let C/K be a (smooth, projective, geometrically connected) curve of genus 2 over an arbitrary field K. In this section we briefly review some of the basic properties of elliptic subcover f : C → E of C/K; cf. [4], [5] and [9].

An elliptic subcover of C/K is a finite K-morphism f : C → E to an elliptic curve E/K which does not factor over a non-trivial isogeny of E/K. If f has the property that its base-change f\overline{K} : C\overline{K} → E\overline{K} to the algebraic closure \overline{K} of K is an elliptic subcover, then we call f a geometric elliptic subcover. Clearly, each geometric elliptic subcover is an elliptic subcover, but the converse assertion is more subtle. However, it turns out that these two concepts are indeed equivalent, as was shown in [9]; cf. Corollary 8 below.

For much of what follows, it is important to note that this condition on f is equivalent to the condition that the induced map f^{∗} : J_{E} → J_{C} on the Jacobians is a closed immersion. Here and below, we use the notations and basic facts about Jacobians as summarized the appendix of [5].

Proposition 6 If f : C → E be a finite morphism to an elliptic curve E/K, then f is a geometric elliptic subcover if and only f^{∗} : J_{C} → J_{E} is a closed immersion. If this is the case, then f is separable.

Proof. The first assertion follows immediately from [4], Proposition 4.2, because (f\overline{K})^{∗} = (f^{∗})\overline{K} is a closed immersion if and only if f^{∗} is a closed immersion. The second assertion follows from the fact that if f is inseparable, then f\overline{K} factors over an inseparable isogeny of E\overline{K}; cf. [9], Proposition 2.2, for the details. □

As in the introduction (or as in [4]), two (geometric) elliptic subcovers f_{i} : C → E_{i} are said to be equivalent if there exists an isomorphism (of curves) φ : E_{1} →^{∼} E_{2} such that f_{2} = φ ∘ f_{1}. We then have the following general fact which generalizes Corollary 4.3 of [4].

Theorem 7 The rule f ↦ f^{∗}J_{E} defines a bijection

Φ_{C} : E(C) →^{∼} S(J_{C})

between the set E(C) of equivalence classes of geometric elliptic subcovers of C and the set S(J_{C}) of elliptic subgroups E ≤ J_{C} of J_{C}.

Proof. (Sketch; details in [9]). The injectivity of Φ_{C} follows easily from Lemma 7.2 of [5]; cf. [9]. However, to prove that Φ_{C} is surjective is much harder, except in special cases. (For example, if C/K has a divisor of degree 1, then a variant of the proof of [4], Corollary 4.3, shows that the map is surjective.) The general case follows from [9], Theorem 1.1, which uses a descent argument based on the existence of normalized covers. Note that if char(K) ≠ 2, then this was already proven in [5]. □

Corollary 8 If f : C → E is a finite morphism to an elliptic curve E/K, then f factors over a geometric elliptic subcover f′ : C → E′ which is unique up to equivalence. In particular, each elliptic subcover f : C → E is a geometric elliptic subcover.

Proof. (Sketch; details in [9]). Put H = Im(f^{∗}) ≤ J_{C}. By Theorem 7 there is a geometric elliptic cover f′ : C → E′ such that (f′)^{∗}J_{E} = H, and it is easy to see that f = ν ∘ f′, for some isogeny ν : E′ → E; cf. [9], Corollary 1.2. □

Remark 9 It follows from Theorem 3.2(f) of [5] that the above bijection Φ_{C} takes elliptic subcovers f : C → E of degree N to elliptic subgroups of degree N with respect to the canonical polarization λ_{C} : J_{C} → Ĵ_{C} in the sense of [5]; cf. equation (2) below.

We now explain how an elliptic subcover f : C → E gives rise to a “presentation” (E, E_{f}^{⊥}, ψ_{f}) of C/K which will be used in the sequel. In view of the bijection of Theorem 7, this is a special case of a “presentation” attached to an elliptic subgroup E ≤ J of a principally polarized abelian surface (J, λ), so we will give the definition in this generality. For this, recall from [8] that if (A, λ) is any principally polarized abelian variety, then for every integer N ≥ 2 we have the pairing (of K-group schemes)

e_{N}^{λ} := e^{Nλ} : A[N] × A[N] → μ_{N}.

Note that when char(K) ∤ N, then we have by formula (5) on p. 232 of [16] that e_{N}^{λ} = \overline{e}_{N} ∘ (1 × λ), where \overline{e}_{N} : A[N] × Â[N] → μ_{N} is the usual canonical pairing.

Definition. If N ≥ 2, then an N-presentation over K is a triple (E, E′, ψ) where E/K and E′/K are elliptic curves and ψ : E[N] → E′[N] is a K-isomorphism of finite group schemes which is an anti-isometry with respect to the e_{N}-pairings.

Moreover, if (J, λ) is a principally polarized abelian surface over K, then a presentation of (J, λ) of degree N over K is a 4-tuple (E, E′, ψ, π) such that (E, E′, ψ) is an N-presentation over K and π : E × E′ → J is an isogeny such that

(1)  Ker(π) = Graph(−ψ) and π̂ ∘ λ ∘ π = N(λ_{E} ⊗ λ_{E′}),

where λ_{E} : E → Ê is the canonical principal polaration of E/K and λ_{E} ⊗ λ_{E′} denotes the product polarization on E × E′, i.e., λ_{E} ⊗ λ_{E′} = ϕ_{θ_{E}+θ_{E′}}, where θ_{E} = θ_{E}^{E×E′} = pr_{E}^{∗}(0_{E}) and θ_{E′} = θ_{E′}^{E×E′} = pr_{E′}^{∗}(0_{E′}), and pr_{E} = pr_{E}^{E×E′} : E × E′ → E and pr_{E′} = pr_{E′}^{E×E′} : E × E′ → E′ are the projections.

We also say that (E, E′, ψ) is an N-presentation of (J, λ) over K if there is some isogeny π such that (E′, E′, ψ, π) is an N-presentation of (J, λ) over K.

In addition, two N-presentations (E_{1}, E′_{1}, ψ_{1}, π_{1}) and (E_{2}, E′_{2}, ψ_{2}, π_{2}) of (J, λ) are said to be isomorphic if there exist isomorphisms α : E_{1} →^{∼} E_{2} and α′ : E′_{1} →^{∼} E′_{2} of ellipic curves such that π_{1} = π_{2} ∘ (α × α′).

The following result clarifies the relation between N-presentations of (J, λ) and elliptic subgroups of J of degree N.

Proposition 10 (a) Let (J, λ) be a principally polarized abelian surface and let E ≤ J be an elliptic subgroup of J with inclusion morphism i_{E} : E ↪ J. If E has λ-degree N, i.e., if

(2)  î_{E} ∘ λ ∘ i_{E} = Nλ_{E},

then there is unique elliptic subgroup E^{⊥} = E_{λ}^{⊥} ≤ J of λ-degree N such that

(3)  ϕ_{E+E^{⊥}} = Nλ,

and there is a unique anti-isometry ψ_{E} = ψ_{J,λ,E} : E[N] →^{∼} E^{⊥}[N] such that

(4)  i_{E^{⊥}} ∘ ψ_{E} = (i_{E})_{|E[N]}.

In addition, π_{E} = π_{J,λ,E} := i_{E} ∘ pr_{E} + i_{E^{⊥}} ∘ pr_{E^{⊥}} : E × E^{⊥} → J is an isogeny such that (E, E^{⊥}, ψ_{E}, π_{E}) is an N-presentation of (J, λ) over K such that π_{E}(θ_{E^{⊥}}) = E.

(b) Let (E, E′, ψ) be an N-presentation over K, and let π_{ψ} : A := E ×E′ → A_{ψ} := A/Graph(−ψ) be the quotient map. Then A_{ψ} has a unique principal polarization λ_{ψ} such that (E, E′, ψ, π_{ψ}) is an N-presentation of (A_{ψ}, λ_{ψ}).

In addition, Ē := π_{ψ}(θ_{E′}) ≃ E and Ē′ = π_{ψ}(θ_{E}) ≃ E′ are elliptic subgroups of A_{ψ} of λ_{ψ}-degree N, and Ē′ = Ē^{⊥}.

(c) If (E, E′, ψ) is an N-presentation of (J, λ), then there is an isomorphism (of polarized abelian surfaces) (J, λ) ≃ (A_{ψ}, λ_{ψ}), where the latter pair is as in part (b).

(d) If (E, E′, ψ, π) is an N-presentation of (J, λ), then \overline{E} := π(θ_{E′}^{E×E′}) is an elliptic subgroup of J of λ-degree N, and the N-presentation (\overline{E}, \overline{E}^{⊥}, ψ\overline{E}, π\overline{E}) of part (a) is isomorphic to (E, E′, ψ, π).

(e) The rule (E, E′, ψ, π) ↦ π(θ_{E′}^{E×E′}) defines a bijection between the set P_{N}(J, λ) of isomorphism classes of N-presentations of (J, λ) and the set S_{N}(J, λ) of elliptic subgroups of J of λ-degree N.

Remark 11 (a) If λ = ϕ_{θ}, for some theta-divisor θ ∈ Div(J), then the left hand side of (2) equals ϕ_{i_{E}^{∗}θ}, so (2) is equivalent to the condition that (E.θ) = N.

Similarly, condition (3) is equivalent to the condition that E + E^{⊥} ≡ Nθ, where ≡ denotes numerical equivalence, and the second condition of (1) is equivalent to the condition that π^{∗}θ ≡ θ_{E} + θ_{E′}.

(b) As the above result shows, the concept of a presentation (E, E′, ψ) is closely related to the “basic construction” of [2] and [5], §5.

(c) If (E, E′, ψ, π) is an N-presentation of (J, λ), and if τ : E′ × E → E × E′ denote the isomorphism defined by τ(x, y) = (y, x), then (E′, E, ψ^{−1}, π ∘ τ) is also an N-presentation of (J, λ) because τ^{∗}(θ_{E}^{E′×E} + θ_{E′}^{E′×E}) = θ_{E}^{E×E′} + θ_{E′}^{E×E} and Ker(π ∘ τ) = Graph(−ψ^{−1}), as is easy to see.

Proof of Proposition 10. (a) Put E^{⊥} = Ker(î_{E} ∘ λ). Then by [5], Proposition 5.2 and Corollary 5.3, E^{⊥} is an elliptic subgroup of J of λ-degree N, and there is a unique isomorphism ψ such that (3) holds. Moreover, ψ is an anti-isometry by [5], Corollary 5.6. By [5], Proposition 5.5 and Corollary 5.6, the morphism π is an isogeny which satisfies (1), so (E, E^{⊥}, ψ, π) is an N-presentation of (A, λ).

It remains to show that π_{E}(θ_{E^{⊥}}) = E, that E^{⊥} satisfies (3) and that this uniquely characterizes E^{⊥}. To prove the latter, suppose that E′ is another elliptic subgroup of J such that ϕ_{E+E′} = Nλ. Then E + E^{⊥} ≡ E + E′, so E^{⊥} ≡ E′. Then by [4], Theorem 2.8, it follows that their base changes E⊥ ; \overline{K} and E′ ; \overline{K} to \overline{K} are equal, and hence E^{⊥} = E′. This proves that E^{⊥} is uniquely determined by (3).

To prove that (3) holds for E^{⊥} as defined above, write A = E × E^{⊥} and let i_{E}^{A} : E ↪ A and i_{E^{⊥}}^{A} : E^{⊥} ↪ A be the canonical inclusions. Then by the definition of π_{E} we have

(5)  π_{E} ∘ i_{E}^{A} = i_{E} and π_{E} ∘ i_{E^{⊥}}^{A} = i_{E^{⊥}},

and so π(θ_{E^{⊥}}^{A}) = i_{E}(E) = E because θ_{E^{⊥}}^{A} = pr_{E^{⊥}}^{∗}(0) = E × {0} = i_{E}^{A}(E). Moreover, π_{|θ_{E^{⊥}}^{A}} : θ_{E^{⊥}}^{A} → E is an isomorphism because θ_{E^{⊥}}^{A} ∩Graph(−ψ) = {0} (as subschemes). Thus, π_{∗}θ_{E^{⊥}}^{A} = E (equality as divisors). Similarly, since θ_{E}^{A} = i_{E^{⊥}}^{A}(E^{⊥}), we see that also π_{∗}θ_{E} = E^{⊥}. Thus, since π is an isogeny of degree N^{2}, it follows that

(6)  π^{∗}E ≡ N^{2}θ_{E^{⊥}} and π^{∗}E^{⊥} ≡ N^{2}θ_{E}.

From this we see that π̂_{E} ∘ ϕ_{E+E^{⊥}} ∘ π_{E} = ϕ_{π_{E}^{∗}(E+E^{⊥})} = N^{2}ϕ_{θ_{E}+θ_{E^{⊥}}} = N^{2}(λ_{E} ⊗ λ_{E^{⊥}}) = Nπ̂_{E} ∘ λ ∘ π, where the latter equality follows from (1). Thus, since π_{E} and π̂_{E} are isogenies, we see that (3) holds. Moreover, as was shown above, we have that π_{E}(θ_{E^{⊥}}) = E.

(b) The existence of the quotient A_{ψ} and of λ_{ψ} satisfying (1) follows from [5], Proposition 5.7, so (E, E′, ψ, π_{ψ}) is an N-presentation of (A_{ψ}, λ_{ψ}).

To prove the second statement, note that by Corollary 5.9 of [5] there exist elliptic subgroups \overline{E}, \overline{E}′ ∈ S_{N}(A_{ψ}, λ_{ψ}) such that

π_{ψ} ∘ i_{E}^{A} = i\overline{E} and π_{ψ} ∘ i_{E′}^{A} = i\overline{E}′.

Thus, by a similar argument as in part (a) we see that π_{|θ_{E}} : θ_{E} → Ē′ and π_{|θ_{E′}} : θ_{E′} → Ē are isomorphisms, where π = π_{ψ}, so E′ ≃ θ_{E} ≃ Ē′, and E ≃ θ_{E′} ≃ Ē and π_{∗}θ_{E} = Ē′ and π_{∗}θ_{E} = Ē′. Thus, as in part (a), it follows that (the analogue of) (3) holds and so Ē′ = Ē^{⊥}.

(c) By hypothesis, there exists π such that (E, E′, ψ, π) is an N-presentation of (J, λ). By (1) and the universal property of quotients there is a unique isomorphism α : A_{ψ} →^{∼} J such that π = α ∘ π_{ψ}. Moreover, since α̂ ∘ λ ∘ α and λ_{ψ} both satisfy the second identity of (1), it follows that α̂ ∘ λ ∘ α = λ_{ψ}. This proves the assertion.

(d) Put A = E × E′. By (1) we know that π : A → J is a quotient of A by Graph(−ψ), so we can identify J with A_{ψ}, and then λ_{ψ} = λ. Thus, by the proof of part (b) we see that \overline{E} := π(θ_{E′}^{A}) ∈ S_{N}(J, λ). Moreover, we know that \overline{E}′ := π(θ_{E}^{A}) ∈ S_{N}(J, λ) and that \overline{E}^{⊥} = \overline{E}′. In addition, we have seen that there exist isomorphisms α : E → \overline{E} and α′ : E′ → \overline{E}′ = \overline{E}^{⊥} such that i\overline{E} ∘ α = π ∘ i_{E}^{A} and α′ ∘ i\overline{E}′ = π ∘ i_{E′}.

To show that π\overline{E} ∘ (α × α′) = π, put \overline{A} = \overline{E} × \overline{E}′ = \overline{E} × \overline{E}^{⊥}. Then by (5)

we have that π\overline{E} ∘ (α × α′) ∘ i_{E}^{A} = π\overline{E} ∘ i\overline{A} ; \overline{E} ∘ α = i\overline{E} ∘ α = π ∘ i_{E}^{A}, and similarly

π\overline{E} ∘ (α × α′) ∘ i_{E′}^{A} = π ∘ i_{E′}^{A}, and so it follows that π\overline{E} ∘ (α × α′) = π, which proves the assertion.

(e) If P_{1} = (E_{1}, E′_{1}, ψ_{1}, π_{1}) is an N-presentation of (J, λ), then by part (d) we see that φ(P_{1}) := π_{1}(θ_{E′_{1}}^{E_{1}×E′_{1}}) ∈ S_{N}(J, λ). Moreover, if (E_{2}, E′_{2}, ψ_{2}, π_{2}) is an Npresentation of (J, λ) which is isomorphic to P_{1} via isomorphisms α : E_{1} → E_{2} and α′ : E′_{1} → E′_{2}, then φ(P_{2}) = φ(P_{1}) because (α × α′)(θ_{E_{1}}^{E_{1}×E′_{1}}) = θ_{E_{2}}^{E_{2}×E′_{2}}, and so the given rule defines a map φ : P_{N}(J, λ) → S_{N}(J, λ).

On the other hand, the rule E ↦ (E, E^{⊥}, ψ_{E}, π_{E}) defines by part (a) a map φ′ : S_{N}(J, λ) → P_{N}(J, λ) such that φ(φ′(E)) = E. Since φ′ ∘ φ = 1_{P_{N} (J,λ)} by part(d), it follows that φ and φ′ are inverses of each other, and so both are bijections. □

Corollary 12 If f : C → E be an elliptic subcover of C/K of degree N, then (f^{∗}J_{E})^{⊥} = Ker(f_{∗}), and there exists an elliptic subcover f^{⊥} : C → E^{⊥} of degree N such that (f^{⊥})^{∗}J_{E^{⊥}} = (f^{∗}J_{E})^{⊥}, and a unique anti-isometry

ψ_{f} : E[N] →^{∼} E_{f}^{⊥}[N] such that (f^{⊥})^{∗} ∘ λ_{E_{f}^{⊥}}ψ_{f} = (f^{∗} ∘ λ_{E})_{|E[N]},

where λ_{E} : E → J_{E} = Ê is the canonical isomorphism. Moreover, if A_{f} = E × E_{f}^{⊥} denotes the product surface, and if

π_{f} := (f^{∗} ∘ λ_{E} ∘ pr_{E}) + ((f^{⊥})^{∗} ∘ λ_{E_{f}^{⊥}}pr_{E_{f}^{⊥}}) : A_{f} = E × E_{f}^{⊥} → J_{C},

then (E, E_{f}^{⊥}, ψ_{f}, π_{f}) is an N-presentation of (J_{C}, λ_{C}) over K. In addition, if

π′_{f} := (λ_{E}^{−1} ∘ f_{∗}, λ_{E_{f}^{⊥}}^{−1} ∘ (f^{⊥})_{∗}) : J_{C} → E × E_{f}^{⊥},

then π′_{f} ∘ π_{f} = [N]_{E×E_{f}^{⊥}}.

Proof. Since f_{∗} = λ_{E}^{−1} ∘ (f^{∗})^{∧} ∘ λ, it follows from the proof of Proposition 10 that (f^{∗}J_{E})^{⊥} = Ker(f_{∗}), and so this has degree N. By Theorem 7 and Remark 9 there exists an elliptic subcover f^{⊥} : C → E_{f}^{⊥} of degree N such that (f^{⊥})^{∗}J_{E^{⊥}} = (f^{∗}J_{E})^{⊥}. Thus Proposition 10 shows that (E, E_{f}^{⊥}, ψ_{f}, π_{f}) is an N-presentation of (J_{C}, λ_{C}).

The last assertion follows immediately from the identities f_{∗} ∘ f^{∗} = [N]_{J_{E}}, (f^{⊥})_{∗} ∘ (f^{⊥})^{∗} = [N]_{J_{E^{⊥}_{f}}}, f_{∗} ∘ (f^{⊥})^{∗} = 0 (by construction) and (f^{⊥})_{∗} ∘ f^{∗} = 0 (by duality). □

Remark 13 If f_{i} : C → E_{i}, i = 1, 2, are two elliptic subcovers of C/K, then the above proof shows that f_{2}^{∗}J_{E_{2}} = (f_{1}^{∗}(J_{E_{1}}))^{⊥} if and only if (f_{1})_{∗}f_{2}^{∗} = 0. (Use the fact that a containment of elliptic subgroups implies equality.)

3 The refined Humbert invariant

As was mentioned in the introduction, the refined Humbert invariant q_{C} of C (cf. [4], [8]) can be used to give a description of the set E(C)) elliptic subcovers of C. Unfortunately, the definition of q_{C} given in [4] and [8] depends on the existence of a theta-divisor θ_{C} ∈ Div(J_{C}) and hence cannot readily be used over an arbitrary ground field. In particular, the Néron-Severi group NS(J_{C}) of J_{C}, which played an important role in the definition of q_{C}, is too small for our purpose.

The simplest way to remedy this problem is to replace NS(J_{C}) by a slightly larger group NS′(J_{C}) which is a subgroup of NS(J_{C} ⊗\overline{K}), where \overline{K} is an algebraic closure of K. Another method consists of replacing NS(J_{C}) by the much more intrinsic group End_{λ}(J_{C}) ≤ End(J_{C}). This leads to the same results because these two groups are naturally isomorphic; cf. Proposition 14 below. However, the second group is more useful in proofs.

To define these groups, let A/K be an arbitrary abelian variety. For any field extension L/K we have an injective homomorphism δ_{L/K} : Div(A) ↪ Div(A_{L}) of the divisor groups, where A_{L} = A ⊗ L is the base-change of A by L, and this induces an injection

\overline{δ}_{L/K} : NS(A) ↪ NS(A_{L}),

where, as usual, NS(A) = Div(A)/ ≡ denotes the group of divisors on A modulo numerical equivalence. Fix an algebraic closure \overline{K} of K, and let

NS′(A) = \{D ∈ NS(A\overline{K}) : nD ∈ \overline{δ}_{\overline{K}/K}(NS(A)), for some n ≥ 1\}

denote the saturation of \overline{δ}_{\overline{K}/K}(NS(A)) in NS(A\overline{K}).

On the other hand, if A has a principal polarization λ : A →^{∼} Â (defined over K), then we can define the additive subgroup End_{λ}(A) of the ring End(A) = End_{K}(A) of K-endomorphisms of A by:

End_{λ}(A) = {α ∈ End(A) : α̂ ∘ λ = λ ∘ α}, = {α ∈ End(A) : α′ = α},

where α′ = r_{λ}(α) := λ^{−1} ∘ α̂ ∘ λ. Thus, End_{λ}(A) consists of those endomorphisms which are symmetric with respect to the Rosati involution r_{λ} defined by λ.

To see that these two groups are isomorphic, recall that there is a natural injective group homomorphism

Φ_{λ} : NS(A) ↪ End_{λ}(A)

which is given by the rule Φ_{λ}(D) = λ^{−1}∘ϕ_{L(D)}; cf. Mumford[16] or §11 of [8]. Note that this isomorphism is clearly compatible with base-change, i.e., if L/K is an extension field, then we have

(7)  Φ_{λ_{L}}(\overline{δ}_{L/K}(D)) = β_{L/K}(Φ_{λ}(D)), for all D ∈ NS(A),

where β_{L/K}(h) = h_{L} ∈ End(A_{L}) is the base-change of the endomorphism h ∈ End(A). We now show:

Proposition 14 (a) If L/K is an extension field, then the base-change homomorphism β_{L/K} : End_{λ}(A) ↪ End_{λ_{L}}(A_{L}) is injective and its cokernel is torsionfree.

(b) There is a unique isomorphism

Φ′_{λ} : NS′(A) →^{∼} End_{λ}(A)

such that the restriction of Φ_{λ\overline{K}} to NS′(A) equals β_{\overline{K}/K} ∘ Φ′_{λ}. Moreover, the quotient NS′(A)/\overline{δ}_{\overline{K}′/K}(NS(A)) is a finite 2-group.

Proof. (a) The fact that β_{L/K} is injective is clear. Next, recall from [6], Lemma 14(b), that β_{L/K}(End(A)) is a primitive submodule of End(A_{L}), i.e., the quotient End(A_{L})/β_{L/K}(End(A)) is torsionfree. Thus, since End_{λ}(A) is primitive in End(A), it follows that β_{L/K}(End_{λ}(A)) is primitive in End(A_{L}) and hence a fortiori in End_{λ_{L}}(A_{L}). This proves the second assertion.

(b) It clearly suffices to show that Φ_{λ\overline{K}}(NS′(A)) = β_{\overline{K}/K}End_{λ}(A) because Φ_{λ\overline{K}} and β_{\overline{K}/K} are both injective. Now since \overline{δ}_{\overline{K}/K}(NS(A)) ⊂ β_{\overline{K}/K}(End_{λ}(A)) by (7) and since the latter subgroup is primitive in End_{λ\overline{K}}(A) by part (a), it follows that that Φ_{λ\overline{K}}(NS′(A)) ⊂ β_{\overline{K}/K}End_{λ}(A).

To prove the opposite inclusion, note first that that there exists θ ∈ NS′(A) such that Φ_{λ\overline{K}}(θ) = 1_{A\overline{K}}. Indeed, since λ is a polarization, there exists θ ∈ NS(A\overline{K}) such that ϕ_{θ} = λ\overline{K}, and so Φ_{λ\overline{K}}(θ) = 1_{A\overline{K}}. Now by Proposition 6.10 of [15] we know that there exists D ∈ NS(A) such that ϕ_{D} = 2λ. Thus \overline{δ}_{\overline{K}/K}(D) = 2θ, and so θ ∈ NS′(A).

Now suppose that h ∈ End_{λ}(A). Since Φ_{λ\overline{K}} is an isomorphism (cf. [16] or [8]), there exists D′ ∈ NS(A\overline{K}) such that Φ_{λ\overline{K}}(D′) = h\overline{K}. Thus ϕ_{D′} = λ′ ; \overline{K}, where λ′ = λ ∘ h. We now show that D′ ∈ NS′(A). For this, note first that since θ is ample, there is an n ≥ 1 such that D′′ := nθ + D′ is ample. Put λ′′ = nλ + λ′. Then λ′′_{\overline{K}′} = nϕ_{θ} + ϕ_{D′} = ϕ_{D′′}, so λ′′ is a polarization of A. Applying Proposition 6.10 of [15] again, we obtain that 2λ′′ = ϕ_{D_{1}}, for some D_{1} ∈ NS(A). Thus \overline{δ}_{\overline{K}/K}(D_{1}) = 2D′′ = 2nθ + 2D′, so D′ ∈ NS(A), as desired.

Finally, since the above proof shows that 2NS′(A) ⊂ \overline{δ}_{\overline{K}/K}(NS(A)), it follows that the quotient is a finite elementary 2-group because NS(A\overline{K}) is finitely generated. □

Remark 15 For any homomorphism h : A_{1} → A_{2} of abelian varieties, the pullback of divisors induces a homomorphism h^{∗} : NS(A_{2}) → NS(A_{1}) of the Néron-Severi groups. Since this map is compatible with the respective base-change maps, it follows that h∗ ; \overline{K} \overline{δ}_{\overline{K}/K}(NS(A_{2})) ⊂ \overline{δ}_{\overline{K}/K}(NS(A_{1})), and so the restriction of h∗ ; \overline{K} to NS′(A_{2}) induces a functorial homomorphism

h^{∗} : NS′(A_{2}) → NS′(A_{1}).

Similarly, the groups End_{λ}(A) have a functorial property which mirrors that of the Néron-Severi groups, as was explained in the appendix of [8], More precisely, if (A_{i}, λ_{i}) are two principally polarized abelian varieties, and if h : A_{1} → A_{2} is a homomorphism, then we put h′ = r_{λ_{1},λ_{2}}(h) := λ_{1}^{−1} ∘ ĥ ∘ λ_{2} : A_{2} → A_{1} and define

h^{♭} = h_{λ_{1},λ_{2}}^{♭} : End(A_{2}) → End(A_{1})

by h^{♭}(α) = h′ ∘ α ∘ h. Since h^{♭} is compatible with the Rosati involutions, i.e.,

(8)  h^{♭}(r_{λ_{2}}(α)) = r_{λ_{1}}(h^{♭}(α)), for all α ∈ End(A_{2}),

we see that h^{♭} restricts to a homomorphism h^{♭} : End_{λ_{2}}(A_{2}) → End_{λ_{1}}(A_{1}). Moreover, we have by [5], equation (61), that

(9)  h^{♭}(Φ_{λ_{2}}(D)) = Φ_{λ_{1}}(h^{∗}D), for all D ∈ NS(A_{2}),

and so it also follows that

(10)  h♭ ; \overline{K}(Φ′_{λ_{2}}(D)) = Φ′_{λ_{1}}(h^{∗}D), for all D ∈ NS′(A_{2}),

For later use, we observe that h′ := r_{λ_{1},λ_{2}}(h) and h are duals of each other in the sense that if h′′ := r_{λ_{2},λ_{1}}(h′), then we have that

(11)  h′′ = h.

This follows immediately from properties of the duality isomorphism κ_{A_{i}} : A_{i} →^{∼} ˆ Â_{i} (cf. [5], p. 42-3). Indeed, h′′ = λ_{2}^{−1} ∘ (λ_{1}^{−1} ∘ ĥ ∘ λ_{2})^{∧} ∘ λ_{1} = λ_{2}^{−1} ∘ λ̂_{2} ∘ ˆ ĥ ∘ λ̂_{1}^{−1} ∘ λ_{1} = κ_{A_{2}}^{−1} ∘ ˆ ĥ ∘ κ_{A_{1}} = h.

We now specialize to the case that A/K is a surface and define the refined Humbert invariant q̃_{(A,λ)} on NS′(A) as the restriction of q̃_{(A\overline{K},λ\overline{K})} to NS′(A), where q̃_{(A\overline{K},λ\overline{K})} denotes the refined Humbert invariant on NS(A\overline{K}) as defined in [4] and [8]. Thus:

(12)  q̃_{(A,λ)} = (D.θ)^{2} − 2(D.D), for D ∈ NS′(A),

where θ ∈ NS(A\overline{K}) is such that ϕ_{θ} = λ\overline{K} and (D_{1}.D_{2}) denotes the intersection number of D_{1}, D_{2} ∈ NS(A\overline{K}). By [4], (3.3) we thus have that

(13)  q̃(D) ≥ 0, ∀ D ∈ NS′(A), and q̃(D) = 0 ⇔ D ∈ ℤθ.

Since q̃(D + nθ) = q̃(D), for all n ∈ ℤ, and since θ ∈ NS′(A), as was shown in the proof of Proposition 14, it follows that q̃ induces a positive-definite quadratic form on

NS(A, λ) := NS′(A)/ℤθ.

Remark 16 If we transport q̃_{(A,λ)} to End_{λ}(A) by using the isomorphism Φ′_{λ} of Proposition 14, then the resulting quadratic form Q̃_{(A,λ)} has a simple direct formula:

Q̃_{(A,λ)} = tr(α^{2}) − \frac{1}{4}(tr(α)^{2}), for α ∈ End_{λ}(A).

Here, tr is the usual trace of an endomorphism as defined in Mumford[16], p. 182.

Since we don’t need this, we won’t prove the above assertion here, i.e., that Q̃_{(A,λ)}(Φ′_{λ}(D)) = q̃_{(A,λ)}(D), for D′ ∈ NS′(A); this is proved in [11].

It is worthwhile to mention that it is shown in [11] that the above definition of Q̃_{(A,λ)} can be generalized to principal polarized abelian varieties (A, λ) of arbitrary dimension, and that one then has that Q_{(A,λ)} defines an integral, positive definite quadratic form on the quotient group 𝔼_{λ}(A) := End_{λ}(A)/ℤ1_{A}.

For later reference, we observe the following compatibility relation of q̃_{(A,λ)} with respect to “polarized isogenies”.

Proposition 17 Let (A_{i}, λ_{i}), i = 1, 2, be two principally polarized abelian surfaces, and let π : A_{1} → A_{2} be an isogeny such that π̂λ_{2}π = Nλ_{1}, for some N ≥ 1. Then π^{∗} : NS′(A_{2}) → NS′(A_{1}) is injective with finite cokernel, and we have

(14)  q̃_{(A_{1},λ_{1})}(π^{∗}D) = N^{2}q̃_{(A_{2},λ_{2})}(D), ∀D ∈ NS′(A_{2}).

Proof. Since π\overline{K} is finite and surjective, it follows that π∗ ; \overline{K} : NS((A_{2})\overline{K}) → NS((A_{1})\overline{K}) is injective (by the projection formula). Thus, its restriction to NS′(A_{2}) is also injective. Moreover, since N^{2}NS′(A_{1}) = [N]^{∗}NS′(A_{1}) = π^{∗}(π′)^{∗}NS′(A_{1}) ⊂ π^{∗}NS′(A_{2}), where N = deg(π) and π′ ∘ π = [N]_{A_{1}}, and since NS′(A_{1}) is finitely generated, we see that the cokernel of π^{∗} is finite.

Let θ_{i} ∈ NS′(A_{i}) be such that ϕ_{θ_{i}} = (λ_{i})\overline{K}, for i = 1, 2. Then the given relation between the λ_{i}’s means that π^{∗}θ_{2} = Nθ_{1}; cf. [5], formula (37). Moreover, the relation implies that deg(π)^{2} = deg([N]_{A_{1}}) = N^{4}, so deg(π) = N^{2}. We thus see that (14) follows directly from the projection formula because with q̃_{i} = q̃_{(A_{i},λ_{i})} we have q̃_{1}(π^{∗}D) = (\frac{1}{N} (π^{∗}θ_{2}.π^{∗}D))^{2} − 2(π^{∗}D.π^{∗}D) = (N(θ_{2}.D))^{2} − 2N^{2}(D.D) = N^{2}q̃_{2}(D). □

We now want to generalize Theorem 3.1 of [4] to principally polarized abelian surfaces (A, λ) which are defined over an arbitrary ground field K. For this, recall from §2 or [5], p. 9, that the λ-degree deg_{λ}(E) of an elliptic subgroup E ≤ A is defined by the rule (2). To state the result, we use the notation cl(D) ∈ NS(A) to denote the class of a divisor D ∈ Div(A) and let cl′(D) = \overline{δ}_{\overline{K}/K}(cl(D)) ∈ NS′(A) be its base-change. We then have:

Theorem 18 Let (A, λ) is a principally polarized abelian surface over an arbitrary field K, and let θ ∈ NS(A\overline{K}) be such that ϕ_{θ} = λ\overline{K}. For any n ≥ 1, the rules

E ↦ cl′(E) and D ↦ D + ℤθ ∈ NS(A, λ)

induce bijections between the following three sets:

(i) the set S_{n}(A, λ) of elliptic subgroups E ∈ S(A) of degree n = deg_{λ}(E); (ii) the set P_{n}(A, λ) of primitive elements D ∈ NS′(A) with (D.D) = 0 and

(D.θ) = n; (iii) the set P_{n^{2}}(q_{(A,λ)}) of primitive elements \overline{D} ∈ NS(A, λ) with q_{(A,λ)}(\overline{D}) = n^{2}; (iv) the set P_{n}(A, λ) of isomorphism classes of N-presentations of (A, λ).

Before proving this, we first establish the following technical fact.

Lemma 19 Let E ∈ S(A) be an elliptic subgroup of A with immersion i_{E} : E ↪ A, and let i′_{E} := r_{λ_{E},λ}(i_{E}) : A → E be its “dual”. Then E^{⊥} := E_{λ}^{⊥} = Ker(i′_{E}) , and (E^{⊥})^{⊥} = E. Moreover,

(15)  Φ_{λ}(cl(E^{⊥})) = e_{E} := i_{E} ∘ i′_{E} and Φ_{λ}(cl(E)) = e_{E^{⊥}},

where we view E and E^{⊥} as divisors on A. In particular,

(16)  Ker(Φ_{λ}(cl(E))) = E.

Proof. The identity E^{⊥} = Ker(i′_{E}) follows from the proof of Proposition 10(a), and the identity (E^{⊥})^{⊥} is clear from (3). To verify (15), note that since E^{⊥} = Ker(i′_{E}) = (i′_{E})^{∗}(0_{E}) by definition, it follows from the functorial formula (61) of [8] (applied to h = i′_{E}) that Φ_{λ}(cl(E^{⊥})) = Φ_{λ}((i′_{E})^{∗}(0_{E})) = i′′_{E}Φ_{λ_{E}}(0_{E})i′_{E} = i_{E}i′_{E} because i′′_{E} = i_{E} and Φ_{λ_{E}}(0_{E}) = λ_{E}^{−1}ϕ_{L(0_{E})} = 1_{E}. This proves the first formula of (15), and the second formula follows from this by replacing E by E^{⊥} and using the identity E = (E^{⊥})^{⊥}.

Since i_{E^{⊥}} is injective, we have Ker(e_{E^{⊥}}) = Ker(i′_{E^{⊥}}) = (E^{⊥})^{⊥} = E. Thus, by (15) we have that Ker(Φ_{λ}(cl(E))) = Ker(e_{E^{⊥}}) = E, which proves (16). □

Proof of Theorem 18. Since the bijection between the sets S_{n}(A, λ) and P(A, λ) was established in Proposition 10, the assertion clearly follows from the following two claims.

Claim 1. The rule E ↦ cl′(E) defines a bijection Ψ_{λ,n} : S_{n}(A, λ) →^{∼} P_{n}(A, λ). If E ∈ S_{n}(A, λ), then \overline{E} := E\overline{K} is an elliptic subgroup of \overline{A} = A\overline{K}, and clearly cl(\overline{E}) = cl′(E) ∈ NS′(A). By Remark 11(a) we have that

(17)  deg_{λ}(E) = (cl′(E).θ).

Clearly (\overline{E}.\overline{E}) = 0; cf. [4], Proposition 2.1. Moreover, since cl′(E) = cl(\overline{E}) is primitive in NS(\overline{A}) by Corollary 2.6 of [4], it is also primitive in NS′(A), and so it follows that cl′(E) ∈ P_{n}(A, λ). Thus, the given rule defines a map Ψ_{λ,n} : S_{n}(A, λ) → P_{n}(A, λ).

It is immediate that Ψ_{λ,n} is injective because if cl′(E) = cl′(E′), then by [4], Theorem 2.8, we have that \overline{E} = \overline{E}′, and so E = E′.

To prove that Ψ_{λ,n} is surjective, let D ∈ P_{n}(A, λ). Since NS′(A) is a primitive subgroup of NS(\overline{A}) by construction, it follows that D is primitive in NS(\overline{A}), and so by Theorem 2.8 of [4] there is an elliptic subgroup \overline{E} of \overline{A} such that cl(\overline{E}) = D. Now since D ∈ NS′(A), we have by Proposition 14 that Φ_{λ\overline{K}}(D) = α\overline{K}, for some α ∈ End_{λ}(A). Then H = Ker(α) is a subgroup scheme of A, and H\overline{K} = Ker(α\overline{K}) = Ker(Φ_{λ\overline{K}}(D)). But by Lemma 19 we have that Ker(Φ_{λ\overline{K}}(D)) = Ē, so H\overline{K} = \overline{E} is an elliptic subgroup of \overline{A}, and hence H is one of A. By (17) we see that deg_{λ}(H) = n, so H ∈ S_{n}(A, λ). This proves Claim 1.

Claim 2. The rule D ↦ \overline{D} = D+ℤθ defines a bijection Ψ′_{λ,n} : P_{n}(A, λ) →^{∼} P_{n^{2}}(q_{(A,λ)}). This is proved in exactly the same method as that of the proof of Theorem 3.1 of [4].

□

We now apply this result to the study of elliptic subcovers of a curve C/K of genus 2.

Theorem 20 Let C/K be a curve of genus 2, and put q_{C} = q_{(J_{C},λ_{C})}. Then the rule f ↦ cl′(f^{∗}J_{E}) + ℤθ_{C}, where ϕ_{θ_{C}} = (λ_{C})\overline{K}, induces for each n ≥ 1 a bijection

ε_{n} : E_{n}(C) →^{∼} P_{n^{2}}(q_{C})

between the set E_{n}(C) of equivalence classes of elliptic subcovers f : C → E of degree n and the set P_{n^{2}}(q_{C}) of primitive elements \overline{D} ∈ NS′(J_{C}, λ_{C}) with q_{C}(\overline{α}) = n^{2}.

Proof. Combine the bijections of Theorem 7 and of Theorem 18. □

4 The isogeny defect

We now study the isogeny defect m_{ψ} of an anti-isometry ψ : E[N] → E′[N]. For technical reasons, it is useful to generalize m_{ψ} (which was defined in the Introduction) as follows.

Definition. Let E/K and E′/K be elliptic curves, and let ψ ∈ Hom(E[N], E′[N]). Moreover, let H ≤ H_{E,E′} := Hom(E, E′) be a primitive subgroup of H_{E,E′}, i.e., H is a subgroup of H_{E,E′} such that H_{E,E′}/H is torsionfree. Then the isogeny defect of ψ with respect to H is

m_{ψ}(H) := min{m ≥ 1 : mψ = h_{|E[N]}, for some h ∈ H}.

Remark 21 (a) Clearly, m_{ψ}(H) = 1 if and only if ψ is induced by an isogeny in H, i.e., if and only if ψ = h_{|E[N]}, for some h ∈ H ≤ Hom(E, E′). Note also that if H = {0}, then m_{ψ}(H) = ord(ψ) by definition.

(b) We can express m_{ψ}(H) as an index of groups. Indeed, let

ρ_{N} : Hom(E, E′) → Hom(E[N], E′[N])

be the restriction map h ↦ h_{|E[N]}, and put H_{N} = ρ_{N}(H), which is a subgroup of Hom(E[N], E′[N]). Then by elementary group theory

(18)  m_{ψ}(H) = [⟨ψ⟩ + H_{N} : H_{N}] = [⟨ψ⟩ : ⟨ψ⟩ ∩ H_{N}].

In particular, we see that m_{ψ}(H) | ord(ψ) | N, and that

(19)  kψ = ρ_{N}(h), for some k ∈ ℤ, h ∈ H ⇒ m_{ψ}(H) | k.

As was mentioned in the introduction, the isogeny defect m_{ψ} := m_{ψ}(Hom(E, E′)) determines the determinant of the refined Humbert invariant q_{C} = q_{(J_{C},λ_{C})} attached to a genus 2 curve C/K with presentation (E, E′, ψ); cf. Theorem 3. While the precise formula will be proven in the next two sections, here we first prove some general facts about m_{ψ}(H) which will be used in the proof of Theorem 3.

Lemma 22 In the above situation we have the exact sequence

0 → NH → H → H_{N} → 0.

Thus, if r = r_{H} := rank(H), then

(20)  |H_{N}| = N^{r} and hence |H_{N} + ⟨ψ⟩| = m_{ψ}(H)N^{r}.

Proof. Clearly, NH ≤ Ker(ρ_{N}) ∩ H. Conversely, if h ∈ Ker(ρ_{N}) ∩ H, then E[N] = Ker([N]_{E}) ≤ Ker(h), so h = h′ ∘ [N]_{E}, for some h′ ∈ Hom(E, E′) (because (E, [N]_{E}) is a quotient of E by E[N]). Thus h = Nh′, and so h′ ∈ H because H is a primitive subgroup of Hom(E, E′) by hypothesis. Thus, h ∈ NH, and so NH = Ker(ρ_{N}) ∩ H. Since ρ_{N}(H) = H_{N} by definition, this shows that the above sequence is exact.

Since H ≃ ℤ^{r}, it follows from the above that H_{N} ≃ (ℤ/Nℤ)^{r}, and so |H_{N}| = N^{r}. Thus, by (18) we obtain that |H_{N} + ⟨ψ⟩| = m_{ψ}(H)|H_{N}| = m_{ψ}(H)N^{r}. □

In order to calculate the discriminant of q_{C}, we shall make use of the trace tr(g) of an element g ∈ End(E[N]). Since tr(g) is closely linked to the transpose g^{t} of g, we first recall the definition of g^{t} and some of its properties. Although this is not necessary, it will be convenient to assume henceforth in this section that char(K) ∤ N.

Proposition 23 If g ∈ Hom(E[N], E′[N]), then there exists a unique homomorphism g^{t} ∈ Hom(E′[N], E[N]) such that

(21)  e_{N}^{E′} ∘ (g × 1_{E′[N]}) = e_{N}^{E} ∘ (1_{E[N]} × g^{t}),

where e_{N}^{E} : E[N] × E[N] → μ_{N} is the usual e_{N}-pairing on E. Moreover, we have

(22)  (g^{t})^{t} = g,
(23)  g^{t}g = [det(g)]_{E[N]},

for all g ∈ Hom(E[N], E′[N]), where det(g) ∈ ℤ/Nℤ is the unique number such that

(24)  e_{N}^{E′} ∘ (g × g) = [det(g)]_{μ_{N}} ∘ e_{N}^{E}.

Proof. The existence and uniqueness of g^{t} follows from the non-degeneracy of e_{N}^{E}. By skew-symmetry we have e_{N}^{E} ∘ ((g^{t})^{t} × 1) = [−1] ∘ e_{N}^{E} ∘ (1 × (g^{t})^{t}) = [−1]e^{E′} ∘ (g^{t} × 1) = e^{E′} ∘ (1 × g^{t}) = e^{E′} ∘ (g × 1), and so (g^{t})^{t} = g by the non-degeneracy of e_{N}^{E}.

The existence of a unique det(g) ∈ ℤ/Nℤ satisfying (24) was explained in [5], §4. Thus, since e_{N}^{E} ∘(1×g^{t}g) = e_{N}^{E} ∘(1×g^{t})∘(1×g) = e_{N}^{E′} ∘(g×1)∘(1×g) = [det(g)]∘e_{N}^{E} = e_{N}^{E} ∘ (1 × [det(g)]), we see that equation (23) follows from the non-degeneracy of e_{N}^{E}.

□

Corollary 24 If g ∈ End(E[N]), then

(25)  g^{t} + g = [tr(g)]_{E[N]},

where the trace tr(g) ∈ ℤ/Nℤ is defined by

tr(g) := det(1 + g) − det(g) − 1.

Proof. Since the transpose is a homomorphism and since [1]^{t} = [1], we obtain from (23) that [tr(g)] = (1 + g)^{t}(1 + g) − g^{t}g − 1 = (1 + g^{t})(1 + g) − g^{t}g − 1 = g^{t} + g, which proves (25). □

Remark 25 (a) If h ∈ Hom(E, E′) is a homomorphism, then its transpose is defined as h^{t} := r_{λ_{E},λ_{E′}}(h) = λ_{E′}^{−1} ∘ ĥ ∘ λ_{E}, where (as before) λ_{E} = ϕ_{0_{E}} : E →^{∼} Ê is the canonical principal polarization of E. By Proposition III.8.2 of [17] we have

(26) (h_{|E[N]})^{t} = (h^{t})_{|E′[N]}, and hence det(h_{|E[N]}) ≡ deg(h) (mod N),

by (23) because h^{t} ∘ h = [deg(h)]_{E}. Note also that if h ∈ End(E), then if follows from (26) and the definitions that tr(h_{|E[N]}) ≡ tr(h) (mod N).

(b) If E′′/K is another elliptic curve and if g_{2} ∈ Hom(E′[N], E′′[N]), then it is easy to see that

(27)  (g_{2} ∘ g_{1})^{t} = g_{1}^{t} ∘ g_{2}^{t}, for all g_{1} ∈ Hom(E[N], E′[N]).

(c) Since the transpose is a homomorphism and [1]^{t} = [1], it follows from (25) that the trace tr : End(E[N]) → ℤ/Nℤ is a homomorphism with

(28)  tr([m]_{E[N]}) ≡ 2m (mod N), for all m ∈ ℤ.

Next we observe that the transpose (and hence det and tr) are compatible when we restrict g ∈ Hom(E[N], E′[N]) to the subgroup E[M], where M|N. More precisely, if π_{N,M}^{E} : E[N] → E[M] is the surjective map induced by multiplication by \overline{N} := \frac{N}{M}, then there exists a unique homomorphism g_{M} = ρ_{N,M}(g) ∈ Hom(E[M], E′[M]) such that

(29)  π_{N,M}^{E′} ∘ g = g_{M} ∘ π_{N,M}^{E}, and hence g_{|E[M]} = ε_{M,N}^{E′} ∘ g_{M},

where ε_{M,N}^{E′} : E′[M] ↪ E′[N] is the canonical inclusion. In particular, we see that ρ_{N,M}(ρ_{N}(h)_{M}) = ρ_{M}(h), for h ∈ Hom(E, E′). Since

(30)  e_{N}^{E} ∘ (1_{E[N]} × ε_{M,N}^{E}) = e_{M}^{E} ∘ (π_{N,M}^{E} × 1_{E[M]})

by [17], Proposition III.8.1(e), it follows immediately that

(31)  (g^{t})_{M} = (g_{M})^{t}, for all g ∈ Hom(E[N], E′[N]).

In particular, from (23) we obtain that

(32)  det(g_{M}) ≡ det(g) (mod M), for all g ∈ Hom(E[N], E′[N]).

5 The determinant of q_{C}

In this section we prove the main structure theorems (Theorems 2 and 3) of the refined Humbert invariant q_{C}. Thus, let C/K be a curve of genus 2 over an arbitrary ground field K, and suppose that (E, E′, ψ) is a presentation of degree N of C/K or, more correctly, of (J_{C}, λ_{C}), in the sense of §2.

The most difficult part of Theorem 2 (respectively, of Theorem 3) is the discriminant formula (i) (respectively, the determinant formula (i′)). As we shall see, both formulae follow easily from the following formula (33) for the determinant of the quadratic form d_{C} = d_{J_{C}} on NS′(J_{C}) which is defined by the intersection pairing, i.e., d_{C}(D) = \frac{1}{2}(D.D), for D ∈ NS′(J_{C}). Note that d_{C}(D) = χ(O_{J_{C}}(D)) ∈ ℤ by the Riemann-Roch Theorem; cf. [16], p. 150.

Theorem 26 Let C/K be curve of genus 2 with presentation (E, E′, ψ) of degree N. If char(K) ∤ N, and if r = rank(Hom(E, E′)) ≥ 1, then the determinant of intersection form d_{C} on NS′(J_{C}) is given by

(33)  det(d_{C}) = (−1)^{r+1}m_{ψ}^{2} det(q_{E,E′}),

where m_{ψ} is the isogeny defect of ψ, and q_{E,E′} is the degree form on Hom(E, E′).

Here, as in [8], the determinant of a quadratic form q on a module M ≃ ℤ^{r} is the determinant of the associated Gram matrix of any basis x_{1}, …, x_{r} of M, i.e.,

det(q) = det((M, q)) = det(G_{q}(x_{1}, …, x_{r})),

where G_{q}(x_{1}, …, x_{r}) = (b_{q}(x_{i}, x_{j}))_{1≤i,j,≤r} is the Gram matrix and b_{q} the associated bilinear form which is given by b_{q}(x, y) = q(x + y) − q(x) − q(y), for all x, y ∈ M.

As we will see presently, Theorem 26 follows almost immediately from the following index formula.

Theorem 27 Let (E, E′, ψ) be an N-presentation over K with associated quotient data (π_{ψ}, A_{ψ}, λ_{ψ}) as in Proposition 10(b). If A = E × E′ and char(K) ∤ N, then

(34)  [NS′(A) : π_{ψ}^{∗}NS′(A_{ψ})] = m_{ψ}N^{ρ},

where ρ = rank(NS′(A_{ψ})) = rank(NS(A_{ψ})) = rank(NS(A)) = rank(Hom(E, E′)) + 2 is the Picard number of A_{ψ} and of A, and m_{ψ} is the isogeny defect of ψ.

We first show how Theorem 26 follows from the Index Theorem 27. For this, we require the following preliminary results.

Lemma 28 If E/K and E′/K are elliptic curves over an arbitrary field K, then the

rule (a, b, h) ↦ α(a, b, h) := ( \begin{matrix}[a]_{E} & h^{t} \\ h & [b]_{E′} \end{matrix} ) defines an isomorphism

α_{E,E′} : H̃_{E,E′} := ℤ × ℤ × Hom(E, E′) →^{∼} End_{λ}(A),

where λ = λ_{E} ⊗ λ_{E′} is the product polarization on A = E × E′. In addition, Φ_{λ} is surjective, and hence we have isomorphisms

Φ_{λ} : NS(A) →^{∼} End_{λ}(A) and \overline{δ}_{\overline{K}/K} : NS(A) →^{∼} NS′(A).

Moreover, if we put D(a, b, h) = Φ_{λ}^{−1}(α_{E,E′}(a, b, h)), for (a, b, h) ∈ H̃_{E,E′}, then

(35)  d_{A}(D(a, b, h)) = ab − deg(h) = ab − q_{E,E′}(h),
(36)  q̃_{(A,λ)}(D(a, b, h)) = (a − b)^{2} + 4 deg(h).

Thus, if r = rank(Hom(E, E′)) ≥ 1, then

(37)  det(d_{A}) = (−1)^{r+1} det(q_{E,E′}).

Proof. Since End_{λ_{E}}(E) = {[m]_{E} : m ∈ ℤ} ≃ ℤ, and similarly End_{λ_{E′}}(E′) ≃ ℤ, the first assertion follows immediately from Proposition 61 of [8]. (Note that this proposition holds for an arbitrary ground field K.)

For (a, b, h) ∈ H̃_{E,E′}, put D̃(a, b, h) := (a − d)θ_{E} + (b − 1)θ_{E′} + Γ_{−h} ∈ Div(A), where d = deg(h), θ_{E} = pr_{E}^{∗}(0_{E}), θ_{E′} = pr_{E′}^{∗}(0_{E′}), and Γ_{−h} denotes the graph of −h. By formula (23) of [8] we have

(38)  Φ_{λ}(cl(D̃(a, b, h))) = α_{E,E′}(a, b, h), for all (a, b, h) ∈ H̃_{E,E′},

and so Φ_{λ} is surjective and hence is an isomorphism because Φ_{λ} is always injective. We thus have that D(a, b, h) = cl(D̃(a, b, h)) and that \overline{δ}_{\overline{K}/K}NS(A) = β_{\overline{K}/K}(End_{λ}(A)) = NS′(A), and so we obtain the indicated isomorphism.

By formula (22) of [8] we obtain that d_{A}(D(a, b, h)) = \frac{1}{2}(D(a, b, h).D(a, b, h)) = ab − deg(h), which is (35). Moreover, since λ = ϕ_{θ}, where θ = D(1, 1, 0), we see that (36) follows from formula (22) of [8] because (D(a, b, h).θ) = a + b.

The last assertion follows from (35) as in [8], Corollary 24. Thus, if f_{1}, …, f_{r} is a basis of H_{E,E′}, and if we put D_{i} = D(0, 0, f_{i}), for 1 ≤ i ≤ r, then θ_{E}, θ_{E′}, D_{1}, …, D_{r} is a basis of NS(A), and so we see from (35) that the Gram matrix G_{d_{A}}(θ_{E}, θ_{E′}, D_{1}, …, D_{r}) of the intersection form d_{A} with respect to this basis is given by the block diagonal matrix

G_{d_{A}}(θ_{E}, θ_{E′}, D_{1}, …, D_{r}) = diag (\binom{0 1}{1 0}, −G_{q}(f_{1}, …, f_{r})) ,

where G_{q}(f_{1}, …, f_{r}) is the Gram matrix of q = q_{E,E′} with respect to the basis f_{1}, …, f_{r}. From this, formula (37) follows by taking the determinant of both sides.

□

Lemma 29 Let A_{1}/K and A_{2}/K be two abelian surfaces and let π : A_{1} → A_{2} be an isogeny of degree N^{2}. If ρ = rank(NS′(A_{1})), then

(39)  det(d_{A_{2}}) = (M/N^{ρ})^{2} det(d_{A_{1}}), where M = [NS′(A_{1}) : π^{∗}(NS′(A_{2}))].

Proof. Put NS_{i} = NS′(A_{i}) and d_{i} = d_{A_{i}}, for i = 1, 2. Viewing π^{∗}NS_{2} as a quadratic subspace of NS_{1} via the intersection form d_{1} on NS_{1}, we see that det((π^{∗}NS_{2}, d_{1})) = M^{2} det((NS_{1}, d_{1})). On the other hand, by the projection formula we have that d_{1}(π^{∗}(D)) = N^{2}d_{2}(D), for all D ∈ NS_{2}, so det((π^{∗}NS_{2}, d_{1})) = N^{2ρ} det((NS_{2}, d_{2})), and hence (39) follows. □

Proof of Theorem 26 (using Theorem 27): Since (J_{C}, λ_{C}) ≃ (A_{ψ}, λ_{ψ}) by Proposition 10(c) and π = π_{ψ} satisfies the hypotheses of Lemma 29, and since M = m_{ψ}N^{ρ} by Theorem 27, we see that (33) follows from (39) and (37) because

det(d_{C}) = (M/N^{ρ})^{2} det(d_{A}) = m_{ψ}^{2} det(d_{A}) = (−1)^{r+1}m_{ψ}^{2} det(q_{E,E′}).
□

Before proving the Index Theorem 27, let us see how the main structure theorem (Theorem 3) follows from Theorem 26. For this, we use the following result which is analogous to Proposition 9 of [8] and which is proved in the same way.

Lemma 30 Let A/K be an abelian surface with principal polarization λ, and let ρ = rank(NS′(A)). Then the determinant of q_{(A,λ)} on NS(A, λ) is given by

(40)  det(q_{(A,λ)}) = \frac{1}{2}(−4)^{ρ−1} det(d_{A}).

Proof of Theorem 3 (using Theorems 26 and 27). By (13) we know that q_{C} is an integral, positive definite quadratic form in t variables, where t = rank(NS(J_{C}, λ)) = ρ − 1, where ρ = rank(NS′(J_{C})). By Theorem 27 we know that ρ = r + 2, where r = rank(Hom(E, E′)), so t = r + 1. This proves the first assertion of Theorem 3.

To prove property (i′), note that by Lemma 30 and Theorem 26 we obtain that

det(q_{C}) = \frac{1}{2}(−4)^{r+1} det(d_{C}) = \frac{1}{2}(−4)^{r+1}(−1)^{r+1}m_{ψ}^{2} det(q_{E,E′}) = 2^{2r+1}m_{ψ}^{2} det(q_{E,E′}).

To prove property (ii), we observe that the hypothesis implies by Proposition 10 that J_{C} has an elliptic subgroup E ≤ J_{C} of λ_{C}-degree N, and so q_{C} represents N^{2} primitively by Theorem 18.

For property (iii) we note that q̃_{C}(D) = (D.θ)^{2} − 4d_{C}(D) ≡ (D.θ)^{2} (mod 4), and so q̃_{C}(D) ≡ 0 or 1 (mod 4), ∀D ∈ End_{λ_{C}}(J_{C}). Thus, property (iii) holds.

Finally, since q̃_{C\overline{K}} does not represent 1 by Proposition 6 of [8], the same is true for its restriction q̃_{C} to NS′(A), and so property (iv) holds. □

Proof of Theorem 4 (using Theorem 26). If m_{ψ} = 1, then by definition ψ = h_{|E[N]} for some h ∈ End(E, E′). Put (as in [1]) ν := \binom{[N] 0}{h [1]} ∈ End(E × E′). Then Ker(ν) = Graph(−ψ), so J_{C} ≃ (E × E′)/Graph(−ψ) ≃ E × E′.

Conversely, suppose that there exists an isomorphism α : A := E × E′ →^{∼} J_{C}. Then r ≥ 1 by [8], Prop. 26, and NS′(A) = α^{∗}NS′(J_{C}) and d_{A}(α^{∗}D) = d_{C}(D), for D ∈ NS′(J_{C}), so by Lemma 28 we obtain that det(d_{C}) = det(NS′(J_{C}), d_{C})) = det(NS′(A), d_{A})) = (−1)^{r+1} det(q_{E,E′}). Comparing this to formula (33) for det(d_{C}) shows that m_{ψ}^{2} = 1, so m_{ψ} = 1, as claimed. □

Proof of Corollary 5 (using Theorems 26 and 27). The first hypothesis implies by Corollary 12 that C/K has an N-presentation (E, E′, ψ) over K, where E′ = E_{f}^{⊥} and ψ = ψ_{f}. By Theorem 27 we have that r = ρ − 2 ≥ 1, so by Theorem 3 (which was proved above) we have that det(q_{C})/2^{2ρ−3} = m_{ψ}^{2} det(q_{E,E′}). Since this is assumed to be squarefree, it follows that m_{ψ} = 1, and so J_{C} ≃ E × E′ by Theorem 4. □

Proof of Theorem 2 (using Theorem 3). If q(x, y) = ax^{2} + bxy + cy^{2} is a binary

quadratic form, then disc(q) = b^{2} −4ac = − det ( \begin{matrix}2a & b \\ b & 2c \end{matrix} ) = − det(q). Since here r = 1 and det(q_{E,E′}) = det((2 deg(h))) = 2d, we see that the discriminant formula follows immediately from the determinant formula (i′) of Theorem 3.

Thus, to finish the proof of Theorem 2, we only need to verify that m|N and that gcd(N/m, d) = 1. For this, note first that m = m_{ψ}|N by Remark 21. Next, since here H_{E,E′} = ℤh, we have by definition that ∃k ∈ ℤ such that mψ = kh_{|E[N]}. Since ord(h_{|E[N]}) = |(H_{E,E”})_{N}| = N by Lemma 22, we see that \frac{N}{(k,N)} = ord(kh_{|E[N]}) = ord(mψ) = \frac{N}{m}, and hence m = (k, N). Put k̄ = \frac{k}{m} and N̄ = \frac{N}{m}. Since mE[N] = E[N̄], we see that k̄h_{|E[N̄]} = ψ_{|E[N̄]}, so ρN̄(k̄h) = ψN̄ in the notation of (29). Taking determinants, we obtain by (32) that k̄^{2}d ≡ det(k̄h_{|E[N̄]}) ≡ det(ψN̄) ≡ det(ψ) ≡ −1 (mod N̄), and so (d, N̄) = 1, as desired. □

6 The index formula

We now turn to the proof of the Index Theorem 27. For technical reasons and for greater generality, it is useful to generalize the theorem as follows.

Theorem 31 Let E/K and E′/K be elliptic curves and let ψ ∈ Hom(E[N], E′[N]). Moreover, let G_{ψ} = Graph(−ψ) be the graph of −ψ and let

π = π_{ψ} : A := E × E′ → A_{ψ} := (E × E′)/G_{ψ},

be the associated quotient isogeny. Assume that A_{ψ} has a principal polarization λ_{ψ}. In addition, let H ≤ Hom(E, E′) be a primitive subgroup, and put

H̃ = {D′(a, b, h)) : a, b ∈ ℤ, h ∈ H} ≤ NS′(A),

where D′(a, b, h) = \overline{δ}_{\overline{K}/K}D(a, b, h) = D(a, b, h\overline{K}). If char(K) ∤ m_{ψ}(H), then

(41)  [H̃ : π_{ψ}^{∗}(NS′(A_{ψ})) ∩ H̃] = m_{ψ}(H)N^{r_{H}+2},

where r_{H} = rank(H) and m_{ψ}(H) is the isogeny defect of ψ with respect to H.

Proof of Theorem 27 (using Theorem 31). Since ψ is an anti-isometry, we know by Proposition 10(b) that there is principal polarization λ_{ψ} on A_{ψ}, and so we can apply Theorem 31 with H = Hom(E, E′). Then H̃ = \overline{δ}_{\overline{K}/K}(NS(A)) = NS′(A) by Lemma 28, so (41) implies (34) with ρ = r_{H} + 2 = rank(NS′(A)) = rank(NS(A)). Moreover, since π_{ψ}^{∗} is injective (cf. Proposition 17), we see from (34) that rank(NS′(A)) = rank(NS′(A_{ψ})). Finally, since by Proposition 14 we know that rank(NS(A_{ψ})) = rank(NS′(A_{ψ})), the rank assertions follow. □

We now turn to the proof of Theorem 31. If m_{ψ}(H) = 1, then (41) follows easily from the following result.

Proposition 32 In the situation of Theorem 31 put ν_{N,h} = ( \begin{matrix}[N] & 0 \\ h & [1] \end{matrix} ) ∈ End(A),

where h ∈ H. Then

(42) ν_{N,h}^{∗}(D) = D′(aN^{2} + bd + Nt, b, fN + bh), ∀D = D′(a, b, f) ∈ H̃,

where d = deg(h) and t = tr(h^{t}f), and hence

(43) D ∈ ν_{N,h}^{∗}NS′(A) ⇔ ρ_{N}(f) = ρ_{N}(bh) and φ_{h}(D) ≡ 0 (mod N^{2}),

where φ_{h}(D′(a, b, f)) = a + bd − tr(h^{t}f). Thus,

(44)  H̃ ∩ ν_{N,h}^{∗}NS′(A) = ν_{N,h}^{∗}(H̃),

and hence

(45)  [H̃ : ν_{N,h}^{∗}(H̃)] = N^{r_{H}+2}.

Proof. The formula (42) follows immediately from formula (76) of [8]. Thus, if D = D′(a, b, f) satisfies ρ_{N}(f) = ρ_{N}(bh) and φ_{h}(D) = cN^{2}, then (f − bh)_{E[N]} = 0, so f − bh = f_{0}N with f_{0} ∈ Hom(E, E′), and then D = ν_{N,h}^{∗}(D′(c, b, f_{0})) by (42) because φ_{h}(D) + bd + Ntr(h^{t}f_{0}) = a + 2bd + tr(h^{t}(Nf_{0} − f)) = a + 2bd − tr(bh^{t}h) = a.

Conversely, if D = D′(a, b, f) ∈ ν_{N,h}^{∗}NS′(A), then D = ν_{N,h}^{∗}(D′), for some D′ = D′(a′, b′, f′). Thus by (42) we have that b = b′, and f = Nf′ +bh, so ρ_{N}(f) = ρ_{N}(bh). Moreover, a = N^{2}a′ + bd + Ntr(h^{t}f′), so N^{2}a′ = a − bd − tr(h^{t}(f − bh)) = φ_{h}(D), and hence φ_{h}(D) ≡ 0 (mod N^{2}). This proves (43).

To prove (44), note first that ν_{N,h}^{∗}(H̃) ⊂ H̃ ∩ ν_{N,h}^{∗}NS′(A) by (42). Now suppose conversely that D = D′(a, b, f) ∈ H̃ ∩ ν_{N,h}^{∗}NS′(A). Then by (43) we have that f = bh + Nf′, for some f′ ∈ H_{E,E′} := Hom(E, E′) and φ_{h}(D) = kN^{2}, for some k ∈ ℤ. Thus tr(h^{t}f) = 2bd + Ntr(h^{t}f′) and so Nk^{2} + bd + Ntr(h^{t}f′) = a. Thus D = ν_{N,h}^{∗}D′(k, b, f′) by (42). Now f′ ∈ H because Nf′ = f − bh ∈ H and H is a primitive subgroup of H_{E,E′}. Thus, D′(k, b, f′) ∈ H̃, and so D = ν_{N,h}^{∗}D′(k, b, f′) ∈ ν_{N,h}^{∗}H̃. This proves (44).

Let ρ_{h} : NS′(A) → Hom(E[N], E′[N]) and φ̃_{h} : NS′(A) → ℤ/N^{2}ℤ be defined by ρ_{h}(D′(a, b, f)) = ρ_{N}(f − bh) and by φ̃_{h}(D) = φ_{h}(D) (mod N^{2}), respectively. Then (43) states that

(46)  ν_{N,h}^{∗}NS′(A) = Ker(ρ_{h}) ∩ Ker(φ̃_{h}).

Since ρ_{h}(D′(0, 0, f)) = ρ_{N}(f), we see that ρ_{h}(H̃) = H_{N}, and so

[H̃ : H̃ ∩ Ker(ρ_{h})] = |H_{N}| = N_{H}^{r},

the latter by (20). Moreover, since D′(1, 0, 0) ∈ H̃ ∩ Ker(ρ_{h}) and φ_{h}(D′(1, 0, 0)) = 1, we see that

[H̃ ∩ Ker(ρ_{h}) : H̃ ∩ ν_{N,h}^{∗}NS′(A)] = |φ̃_{h}(H̃ ∩ Ker(ρ_{h}))| = N^{2},

and so (45) follows by using (44). □

The main task in working out the index (41) in the general case is to find a useful characterization of the elements of N_{ψ} := π_{ψ}^{∗}NS′(A_{ψ}) inside N = NS′(A). Now by Remark 15 it is clear that this problem is equivalent to the (easier) problem of characterizing the elements of the group E_{ψ} := π_{ψ}^{♭}End_{λ_{ψ}}(A_{ψ}) inside E := End_{λ}(A). A first step towards this goal is given by the following general result.

Proposition 33 Let (A_{i}, λ_{i}), i = 1, 2 be two principally polarized abelian varieties and let π : A_{1} → A_{2} be an isogeny. Put π′ = λ_{1}^{−1}π̂λ_{2} ∈ Hom(A_{2}, A_{1}). If α ∈ End(A_{1}) and β ∈ Hom(A_{2}, A_{1}), then

(a) α = β_{1} ∘ π, for some β_{1} ∈ Hom(A_{2}, A_{1}) ⇔ Ker(π) ≤ Ker(α).

(b) β = π′ ∘ γ, for some γ ∈ End(A_{2}) ⇔ Ker(π) ≤ Ker(β̂ ∘ λ_{1}).

(c) α ∈ Im(π^{♭}) ⇔ α = β∘π, for some β ∈ Hom(A_{2}, A_{1}), and Ker(π) ≤ Ker(β̂∘λ_{1}).

Proof. (a) This follows from the universal property of quotients, using the fact that (A_{2}, π) is a quotient of A_{1} by the group scheme Ker(π).

(b) We have that β = π′ ∘ γ, for some γ ∈ End(A_{2}) ⇔ β̂ = γ̂ ∘ (π′)^{∧}, for some γ̂ ∈ End(Â_{2}). Since (π′)^{∧} = λ_{2} ∘ π ∘ λ_{1}^{−1} by (11), we see that the given condition is equivalent to β̂ ∘ λ_{1} = γ′ ∘ π, for some γ′ = γ̂ ∘ λ_{2} ⇔ Ker(π) ≤ Ker(β̂ ∘ λ_{1}), the latter by the same reason as in part (a).

(c) Since α ∈ Im(π^{♭}) ⇔ α = π′ ∘ (γ ∘ π), for some γ ∈ End(A_{2}), this follows immediately from part (b). □

Corollary 34 Suppose that α ∈ End_{λ_{1}}(A_{1}). Then α ∈ π^{♭}End_{λ_{2}}(A_{2}) if and only if α = β ∘ π, for some β ∈ Hom(A_{2}, A_{1}) with Ker(π) ≤ Ker(β̂ ∘ λ_{1}).

Proof. This follows immediately from Proposition 33 and the fact that

(47)  π^{♭}End_{λ_{2}}(A_{2}) = π^{♭}End(A_{2}) ∩ End_{λ_{1}}(A_{1}).

To verify (47), note first that π^{♭}End_{λ_{2}}(A_{2}) ≤ π^{♭}End(A_{2}) ∩ End_{λ_{1}}(A_{1}) by (8). Conversely, suppose that α = π^{♭}(β) ∈ End_{λ}(A_{1}), where β ∈ End(A_{2}). Then by (8) again we have that π^{♭}(r_{λ_{2}}(β)) = r_{λ_{1}}(π^{♭}(β)) = π^{♭}(β), and so r_{λ_{2}}(β) = β because π^{♭} is injective (since π and π′ are isogenies). Thus β ∈ End_{λ_{2}}(A_{2}), and so α ∈ π^{♭}End_{λ_{2}}(A_{2}). This proves the opposite inclusion and hence (47). □

Corollary 35 If Ker(π) ≤ A_{1}[N], then NEnd(A_{1}) ≤ Hom(A_{2}, A_{1})π. Thus, if char(K) ∤ N, and if α ∈ End(A_{1}), then

(48)  Nα ∈ π^{♭}End(A_{2}) ⇔ e_{N}^{λ_{1}}(α(x), y) = 1, ∀x, y ∈ Ker(π)(\overline{K}).

Proof. Since Ker(π) ≤ A_{1}[N], there exists π′ : A_{2} → A_{1} such that π′ ∘π = [N]_{A}. Thus Nα = (α ∘ π′) ∘ π ∈ Hom(A_{2}, A_{1})π, which gives the first assertion.

To prove (48), we thus have by Proposition 33(c) that Nα ∈ π^{♭}End(A_{2}) ⇔ Ker(π) ≤ Ker(β̂ ∘ λ_{1}) ⇔ β̂(λ_{1}(x)) = 0, ∀x ∈ Ker(π)(\overline{K}), where β = α ∘ π′. Since β̂(λ_{1}(x)) ∈ Â_{2}[N], it follows from the non-degeneracy of ē_{N} that

β̂(λ_{1}(x)) = 0 ⇔ ē_{N}(y, β̂(λ_{1}(x))) = 1, ∀y ∈ A_{2}[N].

Now since ē_{N}(y, β̂(λ_{1}(x))) = ē_{N}(β(y), λ_{1}(x)) = ē_{N}(α(π′(y)), λ_{1}(x)) (cf. [13], Lemma 16.2(c)) and since π′(A_{2}[N](\overline{K})) = Ker(π)(\overline{K}), we see that (48) follows. □

Corollary 36 In the situation of Theorem 31 let α = α(a, b, f) ∈ E. Then

(49)  α ∈ Hom(A_{ψ}, A)π_{ψ} ⇔ f^{t}ψ = [a]_{E[N]} and ρ_{N}(f) = bψ.

In particular, if m_{ψ}(H) = N, then for H̃′ := Φ′_{λ}(H̃) ≤ End_{λ}(A) we have that

(50)  H̃′ ∩ Hom(A_{ψ}, A)π_{ψ} = NH̃′.

Proof. By Proposition 33(a), we know that α ∈ Hom(A_{ψ}, A)π_{ψ} ⇔ Graph(−ψ) ≤ Ker(α) ⇔ α ∘ γ_{−ψ} = 0, where γ_{−ψ} : E[N] → A[N] is the graph homomorphism of −ψ. Since α ∘ γ_{−ψ} = ([a]_{E[N]} − f^{t}ψ, ρ_{N}(f) − bψ), the assertion (49) follows.

To prove (50), note first that NH̃′ ⊂ H̃′ ∩ Hom(A_{ψ}, A)π_{ψ} by Corollary 35.

Conversely, if α = α(a, b, f) ∈ H̃′ ∩ Hom(A_{ψ}, A)π_{ψ}, then by (49) we have that bψ = ρ_{N}(f) ∈ ρ_{N}(H), so by (19) we have that N|b and hence ρ_{N}(f) = 0. This means that f = Nf_{0} with f_{0} ∈ H by Lemma 22. Thus [a]_{E[N]} = f^{t}ψ = Nf_{0}^{t}ψ = 0, so N|a, and so α = Nα(\frac{a}{N} , \frac{b}{N} , f_{0}) ∈ NH̃′. This proves (50). □

We can use the above results to prove Theorem 31 in the case that m_{ψ}(H) = N. This follows from the following more general result.

Proposition 37 In the situation of Theorem 31, assume also that char(K) ∤ N. If D = D′(a, b, f) ∈ NS′(A), then

(51) ND ∈ π_{ψ}^{∗}NS′(A_{ψ}) ⇔ p_{ψ}(D) := a + b det(ψ) − tr(f^{t}ψ) ≡ 0 (mod N).

Proof. Put α = Φ′_{λ}(D) = α(a, b, f) ∈ E = End_{λ}(A) and p_{ψ}(α) = p_{ψ}(D). Then by (10) we see that ND ∈ π_{ψ}^{∗}NS′(A_{ψ}) ⇔ Nα ∈ E_{ψ} := (π_{ψ})_{λ,λ_{ψ}}^{♭}End_{λ_{ψ}}(A_{ψ}). To characterize the latter condition, we first observe that

(52) e_{N}^{λ}(α(k_{1}), k_{2}) = e_{N}^{E}(x_{1}, x_{2})^{p_{ψ}(α)}, ∀k_{i} = (x_{i}, −ψ(x_{i})) ∈ Ker(π_{ψ})(\overline{K}), i = 1, 2.

Indeed, since α(k_{1}) = (y_{1}, y_{2}) with y_{1} = ax_{1} − f^{t}ψ(x_{1}), y_{2} = f(x_{1}) − bψ(x_{1}), we have e_{N}^{λ}(α(k_{1}), k_{2}) = e_{N}^{E}(y_{1}, x_{2})e_{N}^{E′}(y_{2} − ψ(x_{2}); cf. the proof of Corollary 5.6 of [5]. Thus e_{N}^{λ}(α(k_{1}), k_{2}) = e_{N}(x_{1}, x_{2})^{a}e_{N}(−f^{t}ψ(x_{1}), x_{2})e_{N}(f(x_{1}), −ψ(x_{2}))e_{N}(−bψ(x_{1}) − ψ(x_{2})). Since e_{N}(f(x_{1}), ψ(x_{2})) = e_{N}(x_{1}, f^{t}ψ(x_{2})), and since

e_{N}^{E}(g(x_{1}), x_{2})e_{N}^{E}(x_{1}, g(x_{2})) = e_{N}^{E}(x_{1}, g^{t}(x_{2})+g(x_{2})) = e_{N}^{E}(x_{1}, x_{2})^{tr(g)}, ∀g ∈ End(E[N]),

we thus see that e_{N}^{λ}(α(k_{1}), k_{2}) = e_{N}(x_{1}, x_{2})^{a+b det(ψ)−tr(f^{t}ψ)}. This proves (52).

Thus, by (48) and (52) we obtain that Nα ∈ E_{ψ} ⇔ e_{N}^{λ}(α(k_{1}), k_{2}) = 1, ∀k_{i} ∈ G_{ψ}(\overline{K}) ⇔ e_{N}^{E}(x_{1}, x_{2})^{p_{ψ}(α)} = 1, ∀x_{i} ∈ E[N](\overline{K}) ⇔ p_{ψ}(α) ≡ 0 (mod N) by the nondegeneracy of e_{N}^{E}. This proves (51). □

Corollary 38 If m_{ψ}(H) = N and char(K) ∤ N, then

(53)  H̃ ∩ π_{ψ}^{∗}NS′(A_{ψ}) = N(H̃ ∩ Ker(p_{ψ})),

where p_{ψ} : NS′(A) → ℤ/Nℤ is as in (51). Thus,

(54)  [H̃ : H̃ ∩ π_{ψ}^{∗}NS′(A_{ψ})] = N^{r_{H}+3}.

Proof. Put N_{ψ} := π_{ψ}^{∗}NS′(A_{ψ}). Then by (51) we have that N(H̃ ∩ Ker(p_{ψ})) ≤ H̃ ∩ N_{ψ}. On the other hand, if D ∈ H̃ ∩ N_{ψ}, then α := Φ′_{λ}(D) ∈ H̃′ ∩ E_{ψ}, where as before E_{ψ} = (π_{ψ})^{♭}End_{λ_{ψ}}(A_{ψ}). Since E_{ψ} ⊂ Hom(A_{ψ}, A)π_{ψ}, it follows from (50) that α = Nα′ with α′ ∈ H̃′, and so D = ND′ with D′ ∈ H̃. Thus, by (51) we obtain that D′ ∈ Ker(p_{ψ}), and so D ∈ N(H̃ ∩ Ker(p_{ψ})). This proves (53).

Since [H̃ : H̃ ∩ Ker(p_{ψ})] = |p_{ψ}(H̃)| = N (because p_{ψ}(D′(1, 0, 0)) ≡ 1 (mod N)), we have by (53) that [H̃ : H̃ ∩ N_{ψ}] = [H̃ : NH̃][NH̃ : N(H̃ ∩ Ker(p_{ψ}))] = N^{r_{H}+2}N = N^{r_{H}+3}, which proves (54). □

By using the following result, we can combine the results of Proposition 32 and of Corollary 38 to deduce Theorem 31.

Proposition 39 In the situation of Theorem 31, put m = m_{ψ}(H). Then there exists an h ∈ H such that mψ = mρ_{N}(h) and we have

(55)  ε_{N,m}^{E′} ∘ ψ̄ ∘ π_{N,m}^{E} = ψ − ρ_{N}(h),

for a unique ψ̄ ∈ Hom(E[m], E′[m]). Moreover, if ν := ν_{\frac{N}{m} ,h} ∈ End(A), then ν^{−1}Gψ̄ = G_{ψ}, and hence there a unique isomorphism α_{ψ} : A_{ψ} →^{∼} Aψ̄ such that

(56)  α_{ψ} ∘ π_{ψ} = πψ̄ ∘ ν.

In addition, mψ̄(H) = m_{ψ}(H).

Proof. By definition there exists h′ ∈ H such that mψ = ρ_{N}(h′). Thus, if we put N̄ = \frac{N}{m}, then this implies that ρ_{N}(N̄h′) = N̄mψ = 0, so N̄h′ = Nh, for some h ∈ H by Lemma 22, and hence h′ = mh. This proves the first assertion.

The identity mψ = mρ_{N}(h) implies that ψN̄ = ρN̄(h), where ψN̄ is as in (29), so E[N̄] ≤ Ker(ψ − ρ_{N}(h)). Since (E[m], π_{N,m}^{E}) is the quotient of E[N] by E[N̄], it follows that there exists a unique ψ̄ ∈ Hom(E[m], E′[N]) = Hom(E[m], E′[m]) such that (55) holds.

Next, let ν = ν_{N̄,h} = \binom{[N̄] 0}{h [1]} ∈ End(A), and let γ_{−ψ} be as in the proof of Corollary 36. Then ν ∘ γ_{−ψ} = γ_{−ψ̄} ∘ π_{N,m} because ν((x, −ψ(x)) = (N̄x, h(x) − ψ(x)) = (N̄x, −ψ̄(N̄x)) by (55), and so ν(G_{ψ}) ≤ Gψ̄, or equivalently, G_{ψ} ≤ ν^{−1}Gψ̄. But since deg(ν) = N̄^{2}, we have that |ν^{−1}Gψ̄| = N̄^{2}|Gψ̄| = N̄^{2}m^{2} = N^{2} = |G_{ψ}|, and so G_{ψ} = ν^{−1}Gψ̄, as claimed.

Since Ker(πψ̄ ∘ ν) = ν^{−1}(Gψ̄) = Ker(π_{ψ}) by what was just proved, it follows from the universal property of quotients that there is a unique isomorphism α_{ψ} : A_{ψ} →^{∼} Aψ̄ such that (56) holds.

It remains to prove that m̄ := mψ̄(H) = m. Since ψ̄ ∈ Hom(E[m], E′[m]), it is clear that m̄|m. Now by definition there exists a g ∈ H such that m̄ψ̄ = ρ_{m}(g). Then by (55) we have that m̄(ψ − ρ_{N}(h)) = m̄ψ̄ ∘ π_{N,m} = ρ_{m}(g) ∘ π_{N,m} = N̄ρ_{N}(g), so m̄ψ = ρ_{N}(N̄g + m̄h). Since N̄g + m̄h ∈ H, this means that m|m̄ by (19), and so m̄ = m, as desired. □

Corollary 40 In the situation of Proposition 39, let D ∈ H ∩ π_{ψ}^{∗}NS′(A_{ψ}) and put N̄ = \frac{N}{m}. Then D = mD′, where D′ = D′(a, b, f) ∈ H̃ satisfies

(57)  ρN̄(f) = ρN̄(bh) and φ_{h}(D′) ≡ 0 (mod N̄^{2}).

Proof. By (56), ∃D′′ ∈ NS′(Aψ̄) such that D′ = ν^{∗}π∗ ; ψ̄(D′′). By (44) we see that π∗ ; ψ̄(D′′) ∈ H̃. Then (53) (applied to ψ̄) shows that π∗ ; ψ̄(D′′) = mD̃′, for some D̃′ ∈ H̃. Thus D = mD′, where D′ = ν^{∗}D̃′ ∈ H̃ ∩ ν^{∗}NS′(A), and (57) holds by (43). □

Proof of Theorem 31. Let h, ψ̄, ν and α := α_{ψ} be as in Proposition 39. Put m = m_{ψ}(H) = mψ̄(H) and N̄ = \frac{N}{m}. Since ν = ν_{N̄,h} by Proposition 39, we are in the situation of Proposition 32 and so H̃ ∩ ν^{∗}N = ν^{∗}H̃ by (44), where N = NS′(A).

Since Nψ̄ := π∗ ; ψ̄NS′(Aψ̄) ≤ N, we have that H̃ ∩ ν^{∗}Nψ̄ = H̃ ∩ ν^{∗}N ∩ ν^{∗}Nψ̄ =

ν^{∗}H̃∩ν^{∗}Nψ̄ = ν^{∗}(H̃∩Nψ̄), the latter because ν^{∗} is injective. Since N_{ψ} := π_{ψ}^{∗}NS′(A_{ψ}) = ν^{∗}π∗ ; ψ̄(Aψ̄) by (56), we thus obtain that H̃ ∩ N_{ψ} = H̃ ∩ ν^{∗}Nψ̄ = ν^{∗}(H̃ ∩ Nψ̄), and so

[H̃ : H̃ ∩ N_{ψ}] = [H̃ : ν^{∗}H̃][ν^{∗}H̃ : ν^{∗}(H̃ ∩ Nψ̄)] = [H̃ : ν^{∗}H̃][H̃ : H̃ ∩ Nψ̄].

Now by (45) we have that [H̃ : ν^{∗}H̃] = N̄ r̃, where r̃ = rank(H̃) = r_{H} + 2. Moreover, since ψ̄ ∈ Hom(E[m], E′[m]) and mψ̄ = m, we are in the situation of Corollary 38 (with N replaced by m), and so [H̃ : H̃ ∩ Nψ̄)] = m^{r̃+1} by (54). Thus, [H̃ : H̃ ∩ N_{ψ}] = N̄^{r̃+1}m^{r̃+1} = mN r̃, which proves (41) and hence Theorem 31. □

References

[1] C. Diem, G. Frey, Non-constant curves of genus 2 with infinite pro-Galois covers. Israel Journal of Mathematics 164 (2008), 193–220.

[2] G. Frey, E. Kani, Curves of genus 2 covering elliptic curves and an arithmetical application. In: Arithmetic Algebraic Geometry (G. van der Geer, F. Oort, J. Steenbrink, eds.), Progress In Math. vol. 89, Birkhäuser, Boston, 1991, pp. 153-176.

[3] G. Frey, E. Kani, Curves of genus 2 with elliptic differentials and associated Hurwitz spaces. Contemporary Math. 487 (2009), 33–81.

[4] E. Kani, Elliptic curves on abelian surfaces. Manus. math. 84 (1994), 199–223.

[5] E. Kani, The Hurwitz space of genus 2 covers of an elliptic curve. Collect. Math. 54 (2003), 1–51.

[6] E. Kani, Products of CM elliptic curves. Collect. Math. 62 (2011), 297-339.

[7] E. Kani, Jacobians isomorphic to a product of two elliptic curves and ternary quadratic forms. J. Number Theory 139 (2014), 138–174.

[8] E. Kani, The moduli spaces of Jacobians isomorphic to a product of two elliptic curves. Collect. Math. 67 (2016), 21–54.

[9] E. Kani, Elliptic subcovers of hyperelliptic curves. To appear in Math. Nachr..

[10] E. Kani, Elliptic subcovers of a curve of genus 2. II. The refined Humbert invariant. Preprint, 2017.

[11] E. Kani, Generalized Humbert Varieties. In preparation.

[12] A. Krazer, Lehrbuch der Thetafunktionen. Leipzig, 1903; Chelsea Reprint, New York, 1970.

[13] J.S. Milne, Abelian varieties. In: Arithmetic Geometry. (G. Cornell, J. Silverman, eds.), Springer-Verlag, New York, 1986; pp. 103–150.

[14] J.S. Milne, Jacobian varieties. In: Arithmetic Geometry. (G. Cornell, J. Silverman, eds.), Springer-Verlag, New York, 1986; pp. 165–212.

[15] D. Mumford, Geometric Invariant Theory. Springer-Verlag, Berlin, 1966.

[16] D. Mumford, Abelian Varieties. Oxford U Press, Oxford, 1970.

[17] J. H. Silverman, The Arithmetic of Elliptic Curves. GTM 106, Springer-Verlag, New York, 1986.