Coverage for local/lib/python2.7/site-packages/sage/misc/c3_controlled.pyx : 82%
Hot-keys on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
|
""" The C3 algorithm, under control of a total order
Abstract ========
Python handles multiple inheritance by computing, for each class, a linear extension of the poset of all its super classes (the Method Resolution Order, MRO). The MRO is calculated recursively from local information (the *ordered* list of the direct super classes), with the so-called ``C3`` algorithm. This algorithm can fail if the local information is not consistent; worst, there exist hierarchies of classes with provably no consistent local information.
For large hierarchy of classes, like those derived from categories in Sage, maintaining consistent local information by hand does not scale and leads to unpredictable ``C3`` failures (the dreaded "could not find a consistent method resolution order"); a maintenance nightmare.
This module implements a final solution to this problem. Namely, it allows for building automatically the local information from the bare class hierarchy in such a way that guarantees that the ``C3`` algorithm will never fail.
Err, but you said that this was provably impossible? Well, not if one relaxes a bit the hypotheses; but that's not something one would want to do by hand :-)
The problem ===========
Consider the following hierarchy of classes::
sage: class A1(object): pass sage: class A2(object): ....: def foo(self): return 2 sage: class A3(object): pass sage: class A4(object): ....: def foo(self): return 4 sage: class A5(A2, A1): ....: def foo(self): return 5 sage: class A6(A4, A3): pass sage: class A7(A6, A5): pass
If ``a`` is an instance of ``A7``, then Python needs to choose which implementation to use upon calling ``a.foo()``: that of ``A4`` or ``A5``, but obviously not that of ``A2``. In Python, like in many other dynamic object oriented languages, this is achieved by calculating once for all a specific linear extension of the hierarchy of the super classes of each class, called its Method Resolution Order (MRO)::
sage: [cls.__name__ for cls in A7.mro()] ['A7', 'A6', 'A4', 'A3', 'A5', 'A2', 'A1', 'object']
Thus, in our example, the implementation in ``A4`` is chosen::
sage: a = A7() sage: a.foo() 4
Specifically, the MRO is calculated using the so-called ``C3`` algorithm which guarantees that the MRO respects not only inheritance, but also the order in which the bases (direct super classes) are given for each class.
However, for large hierarchies of classes with lots of multiple inheritance, like those derived from categories in Sage, this algorithm easily fails if the order of the bases is not chosen consistently (here for ``A2`` w.r.t. ``A1``)::
sage: class B6(A1,A2): pass sage: class B7(B6,A5): pass Traceback (most recent call last): ... TypeError: Error when calling the metaclass bases Cannot create a consistent method resolution order (MRO) for bases ...
There actually exist hierarchies of classes for which ``C3`` fails whatever order of the bases is chosen; the smallest such example, admittedly artificial, has ten classes (see below). Still, this highlights that this problem has to be tackled in a systematic way.
Fortunately, one can trick ``C3``, without changing the inheritance semantic, by adding some super classes of ``A`` to the bases of ``A``. In the following example, we completely force a given MRO by specifying *all* the super classes of ``A`` as bases::
sage: class A7(A6, A5, A4, A3, A2, A1): pass sage: [cls.__name__ for cls in A7.mro()] ['A7', 'A6', 'A5', 'A4', 'A3', 'A2', 'A1', 'object']
Luckily this can be optimized; here it is sufficient to add a single base to enforce the same MRO::
sage: class A7(A6, A5, A4): pass sage: [cls.__name__ for cls in A7.mro()] ['A7', 'A6', 'A5', 'A4', 'A3', 'A2', 'A1', 'object']
A strategy to solve the problem ===============================
We should recall at this point a design decision that we took for the hierarchy of classes derived from categories: *the semantic shall only depend on the inheritance order*, not on the specific MRO, and in particular not on the order of the bases (see the section ``On the order of super categories`` in the :mod:`category primer <sage.categories.primer>`). If a choice needs to be made (for example for efficiency reasons), then this should be done explicitly, on a method-by-method basis. In practice this design goal is not yet met.
.. NOTE::
When managing large hierarchies of classes in other contexts this may be too strong a design decision.
The strategy we use for hierarchies of classes derived from categories is then:
1. To choose a global total order on the whole hierarchy of classes. 2. To control ``C3`` to get it to return MROs that follow this total order.
A basic approach for point 1., that will work for any hierarchy of classes, is to enumerate the classes while they are constructed (making sure that the bases of each class are enumerated before that class), and to order the classes according to that enumeration. A more conceptual ordering may be desirable, in particular to get deterministic and reproducible results. In the context of Sage, this is mostly relevant for those doctests displaying all the categories or classes that an object inherits from.
Getting fine control on C3 ==========================
This module is about point 2.
The natural approach would be to change the algorithm used by Python to compute the MRO. However, changing Python's default algorithm just for our needs is obviously not an option, and there is currently no hook to customize specific classes to use a different algorithm. Pushing the addition of such a hook into stock Python would take too much time and effort.
Another approach would be to use the "adding bases" trick straightforwardly, putting the list of *all* the super classes of a class as its bases. However, this would have several drawbacks:
- It is not so elegant, in particular because it duplicates information: we already know through ``A5`` that ``A7`` is a subclass of ``A1``. This duplication could be acceptable in our context because the hierarchy of classes is generated automatically from a conceptual hierarchy (the categories) which serves as single point of truth for calculating the bases of each class.
- It increases the complexity of the calculation of the MRO with ``C3``. For example, for a linear hierachy of classes, the complexity goes from `O(n^2)` to `O(n^3)` which is not acceptable.
- It increases the complexity of inspecting the classes. For example, the current implementation of the ``dir`` command in Python has no cache, and its complexity is linear in the number of maximal paths in the class hierarchy graph as defined by the bases. For a linear hierarchy, this is of complexity `O(p_n)` where `p_n` is the number of integer partitions of `n`, which is exponential. And indeed, running ``dir`` for a typical class like ``GradedHopfAlgebrasWithBasis(QQ).parent_class`` with ``37`` super classes took `18` seconds with this approach.
Granted: this mostly affects the ``dir`` command and could be blamed on its current implementation. With appropriate caching, it could be reimplemented to have a complexity roughly linear in the number of classes in the hierarchy. But this won't happen any time soon in a stock Python.
This module refines this approach to make it acceptable, if not seamless. Given a hierarchy and a total order on this hierarchy, it calculates for each element of the hierarchy the smallest list of additional bases that forces ``C3`` to return the desired MRO. This is achieved by implementing an instrumented variant of the ``C3`` algorithm (which we call *instrumented ``C3``*) that detects when ``C3`` is about to take a wrong decision and adds one base to force the right decision. Then, running the standard ``C3`` algorithm with the updated list of bases (which we call *controlled ``C3``*) yields the desired MRO.
EXAMPLES:
As an experimentation and testing tool, we use a class :class:`HierarchyElement` whose instances can be constructed from a hierarchy described by a poset, a digraph, or more generally a successor relation. By default, the desired MRO is sorted decreasingly. Another total order can be specified using a sorting key.
We consider the smallest poset describing a class hierarchy admitting no MRO whatsoever::
sage: P = Poset({10: [9,8,7], 9:[6,1], 8:[5,2], 7:[4,3], 6: [3,2], 5:[3,1], 4: [2,1] }, linear_extension=True, facade=True)
And build a `HierarchyElement` from it::
sage: from sage.misc.c3_controlled import HierarchyElement sage: x = HierarchyElement(10, P)
Here are its bases::
sage: HierarchyElement(10, P)._bases [9, 8, 7]
Using the standard ``C3`` algorithm fails::
sage: x.mro_standard Traceback (most recent call last): ... ValueError: Can not merge the items 3, 3, 2.
We also get a failure when we relabel `P` according to another linear extension. For easy relabelling, we first need to set an appropriate default linear extension for `P`::
sage: linear_extension = list(reversed(IntegerRange(1,11))) sage: P = P.with_linear_extension(linear_extension) sage: list(P) [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
Now, we play with the fifth linear extension of `P`::
sage: L = P.linear_extensions() sage: Q = L[5].to_poset() sage: Q.cover_relations() [[10, 9], [10, 8], [10, 7], [9, 6], [9, 3], [8, 5], [8, 2], [7, 4], [7, 1], [6, 2], [6, 1], [5, 3], [5, 1], [4, 3], [4, 2]] sage: x = HierarchyElement(10, Q) sage: x.mro_standard Traceback (most recent call last): ... ValueError: Can not merge the items 2, 3, 3.
On the other hand, both the instrumented ``C3`` algorithm, and the controlled ``C3`` algorithm give the desired MRO::
sage: x.mro [10, 9, 8, 7, 6, 5, 4, 3, 2, 1] sage: x.mro_controlled [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
The above checks, and more, can be run with::
sage: x._test_mro()
In practice, the control was achieved by adding the following bases::
sage: x._bases [9, 8, 7] sage: x._bases_controlled [9, 8, 7, 6, 5]
Altogether, four bases were added for control::
sage: sum(len(HierarchyElement(q, Q)._bases) for q in Q) 15 sage: sum(len(HierarchyElement(q, Q)._bases_controlled) for q in Q) 19
This information can also be recovered with::
sage: x.all_bases_len() 15 sage: x.all_bases_controlled_len() 19
We now check that the ``C3`` algorithm fails for all linear extensions `l` of this poset, whereas both the instrumented and controlled ``C3`` algorithms succeed; along the way, we collect some statistics::
sage: stats = [] sage: for l in L: ....: x = HierarchyElement(10, l.to_poset()) ....: try: # Check that x.mro_standard always fails with a ValueError ....: x.mro_standard ....: except ValueError: ....: pass ....: else: ....: assert False ....: assert x.mro == list(P) ....: assert x.mro_controlled == list(P) ....: assert x.all_bases_len() == 15 ....: stats.append(x.all_bases_controlled_len()-x.all_bases_len())
Depending on the linear extension `l` it was necessary to add between one and five bases for control; for example, `216` linear extensions required the addition of four bases::
sage: Word(stats).evaluation_sparse() [(1, 36), (2, 108), (3, 180), (4, 216), (5, 180)]
We now consider a hierarchy of categories::
sage: from operator import attrgetter sage: x = HierarchyElement(Groups(), attrcall("super_categories"), attrgetter("_cmp_key")) sage: x.mro [Category of groups, Category of monoids, Category of semigroups, Category of inverse unital magmas, Category of unital magmas, Category of magmas, Category of sets, Category of sets with partial maps, Category of objects] sage: x.mro_standard [Category of groups, Category of monoids, Category of semigroups, Category of inverse unital magmas, Category of unital magmas, Category of magmas, Category of sets, Category of sets with partial maps, Category of objects]
For a typical category, few bases, if any, need to be added to force ``C3`` to give the desired order::
sage: C = FiniteFields() sage: x = HierarchyElement(C, attrcall("super_categories"), attrgetter("_cmp_key")) sage: x.mro == x.mro_standard False sage: x.all_bases_len() 70 sage: x.all_bases_controlled_len() 74
sage: C = GradedHopfAlgebrasWithBasis(QQ) sage: x = HierarchyElement(C, attrcall("super_categories"), attrgetter("_cmp_key")) sage: x._test_mro() sage: x.mro == x.mro_standard False sage: x.all_bases_len() 94 sage: x.all_bases_controlled_len() 101
The following can be used to search through the Sage named categories for any that requires the addition of some bases. The output may change a bit when the category hierarchy is changed. As long as the list below does not change radically, it's fine to just update this doctest::
sage: from sage.categories.category import category_sample sage: sorted([C for C in category_sample() ....: if len(C._super_categories_for_classes) != len(C.super_categories())], ....: key=str) [Category of affine weyl groups, Category of fields, Category of finite dimensional algebras with basis over Rational Field, Category of finite dimensional hopf algebras with basis over Rational Field, Category of finite enumerated permutation groups, Category of finite weyl groups, Category of group algebras over Rational Field, Category of number fields]
AUTHOR:
- Nicolas M. Thiery (2012-09): initial version. """ #***************************************************************************** # Copyright (C) 2012-2013 Nicolas M. Thiery <nthiery at users.sf.net> # # Distributed under the terms of the GNU General Public License (GPL) # http://www.gnu.org/licenses/ #******************************************************************************
from sage.misc.classcall_metaclass import ClasscallMetaclass, typecall from sage.misc.cachefunc import cached_function, cached_method from sage.misc.lazy_attribute import lazy_attribute from sage.structure.dynamic_class import dynamic_class
############################################################################## # Implementation of the total order between categories ##############################################################################
cdef tuple atoms = ("FacadeSets", "FiniteSets", "Sets.Infinite", "EnumeratedSets", "SetsWithGrading", "Posets", "LatticePosets", "Crystals", "AdditiveMagmas", "FiniteDimensionalModules", "GradedModules", "ModulesWithBasis", "Magmas", "Semigroups", "Monoids", "PermutationGroups", "MagmasAndAdditiveMagmas", "Rngs", "Domains", "HopfAlgebras")
cdef dict flags = { atom: 1 << i for i,atom in enumerate(atoms) }
cdef class CmpKey: r""" This class implements the lazy attribute ``Category._cmp_key``.
The comparison key ``A._cmp_key`` of a category is used to define an (almost) total order on non-join categories by setting, for two categories `A` and `B`, `A<B` if ``A._cmp_key > B._cmp_key``. This order in turn is used to give a normal form to join's, and help toward having a consistent method resolution order for parent/element classes.
The comparison key should satisfy the following properties:
- If `A` is a subcategory of `B`, then `A < B` (so that ``A._cmp_key > B._cmp_key``). In particular, :class:`Objects() <Objects>` is the largest category.
- If `A != B` and taking the join of `A` and `B` makes sense (e.g. taking the join of ``Algebras(GF(5))`` and ``Algebras(QQ)`` does not make sense), then `A<B` or `B<A`.
The rationale for the inversion above between `A<B` and ``A._cmp_key > B._cmp_key`` is that we want the order to be compatible with inclusion of categories, yet it's easier in practice to create keys that get bigger and bigger while we go down the category hierarchy.
This implementation applies to join-irreducible categories (i.e. categories that are not join categories). It returns a pair of integers ``(flags, i)``, where ``flags`` is to be interpreted as a bit vector. The first bit is set if ``self`` is a facade set. The second bit is set if ``self`` is finite. And so on. The choice of the flags is adhoc and was primarily crafted so that the order between categories would not change too much upon integration of :trac:`13589` and would be reasonably session independent. The number ``i`` is there to resolve ambiguities; it is session dependent, and is assigned increasingly when new categories are created.
.. NOTE::
This is currently not implemented using a :class:`lazy_attribute` for speed reasons only (the code is in Cython and takes advantage of the fact that Category objects always have a ``__dict__`` dictionary)
.. TODO::
- Handle nicely (covariant) functorial constructions and axioms
EXAMPLES::
sage: Objects()._cmp_key (0, 0) sage: SetsWithPartialMaps()._cmp_key (0, 1) sage: Sets()._cmp_key (0, 2) sage: Sets().Facade()._cmp_key (1, ...) sage: Sets().Finite()._cmp_key (2, ...) sage: Sets().Infinite()._cmp_key (4, ...) sage: EnumeratedSets()._cmp_key (8, ...) sage: FiniteEnumeratedSets()._cmp_key (10, ...) sage: SetsWithGrading()._cmp_key (16, ...) sage: Posets()._cmp_key (32, ...) sage: LatticePosets()._cmp_key (96, ...) sage: Crystals()._cmp_key (136, ...) sage: AdditiveMagmas()._cmp_key (256, ...) sage: Magmas()._cmp_key (4096, ...) sage: CommutativeAdditiveSemigroups()._cmp_key (256, ...) sage: Rings()._cmp_key (225536, ...) sage: Algebras(QQ)._cmp_key (225536, ...) sage: AlgebrasWithBasis(QQ)._cmp_key (227584, ...) sage: GradedAlgebras(QQ)._cmp_key (226560, ...) sage: GradedAlgebrasWithBasis(QQ)._cmp_key (228608, ...)
For backward compatibility we currently want the following comparisons::
sage: EnumeratedSets()._cmp_key > Sets().Facade()._cmp_key True sage: AdditiveMagmas()._cmp_key > EnumeratedSets()._cmp_key True
sage: Category.join([EnumeratedSets(), Sets().Facade()]).parent_class._an_element_.__module__ 'sage.categories.enumerated_sets'
sage: CommutativeAdditiveSemigroups()._cmp_key < Magmas()._cmp_key True sage: VectorSpaces(QQ)._cmp_key < Rings()._cmp_key True sage: VectorSpaces(QQ)._cmp_key < Magmas()._cmp_key True """ cdef int count def __init__(self): """ Sets the internal category counter to zero.
EXAMPLES::
sage: Objects()._cmp_key # indirect doctest (0, 0) """ self.count = -1
def __get__(self, object inst, object cls): """ Bind the comparison key to the given instance
EXAMPLES::
sage: C = Algebras(FractionField(QQ['x'])) sage: C._cmp_key (225536, ...) sage: '_cmp_key' in C.__dict__ # indirect doctest True """ # assert not isinstance(inst, JoinCategory) # Note that cls is a DynamicClassMetaclass, hence not a type cdef object cat
_cmp_key = CmpKey()
cdef class CmpKeyNamed: """ This class implements the lazy attribute ``CategoryWithParameters._cmp_key``.
.. SEEALSO::
- :class:`CmpKey` - :class:`lazy_attribute` - :class:`sage.categories.category.CategoryWithParameters`.
.. NOTE::
- The value of the attribute depends only on the parameters of this category.
- This is currently not implemented using a :class:`lazy_attribute` for speed reasons only.
EXAMPLES::
sage: Algebras(GF(3))._cmp_key == Algebras(GF(5))._cmp_key # indirect doctest True sage: Algebras(ZZ)._cmp_key != Algebras(GF(5))._cmp_key True
""" def __get__(self, object inst, object cls): """ EXAMPLES::
sage: Algebras(GF(3))._cmp_key == Algebras(GF(5))._cmp_key # indirect doctest True sage: Algebras(ZZ)._cmp_key != Algebras(GF(5))._cmp_key True
""" pass
_cmp_key_named = CmpKeyNamed()
##############################################################################
def C3_merge(list lists): r""" Return the input lists merged using the ``C3`` algorithm.
EXAMPLES::
sage: from sage.misc.c3_controlled import C3_merge sage: C3_merge([[3,2],[4,3,1]]) [4, 3, 2, 1] sage: C3_merge([[3,2],[4,1]]) [3, 2, 4, 1]
This function is only used for testing and experimenting purposes, but exercised quite some by the other doctests in this file.
It is an extract of :func:`sage.misc.c3.C3_algorithm`; the latter could be possibly rewritten to use this one to avoid duplication. """ # Data structure / invariants: # We will be working with the MROs of the super objects # together with the list of bases of ``self``. # Each list is split between its head (in ``heads``) and tail (in # ``tails'') . Each tail is stored reversed, so that we can use a # cheap pop() in lieue of pop(0). A duplicate of the tail is # stored as a set in ``tailsets`` for cheap membership testing. # Since we actually want comparison by identity, not equality, # what we store is the set of memory locations of objects cdef object O, X cdef list tail, l cdef set tailset
cdef int i, j, nbheads cdef bint next_item_found
# Does O appear in none of the tails? ``all(O not in tail for tail in tailsets)`` # Clear O from other heads, removing the line altogether # if the tail is already empty. # j goes down so that ``del heads[j]`` does not screw up the numbering else: # No head is available
cpdef identity(x): r""" EXAMPLES::
sage: from sage.misc.c3_controlled import identity sage: identity(10) 10 """
cpdef tuple C3_sorted_merge(list lists, key=identity): r""" Return the sorted input lists merged using the ``C3`` algorithm, with a twist.
INPUT:
- ``lists`` -- a non empty list (or iterable) of lists (or iterables), each sorted strictly decreasingly according to ``key`` - ``key`` -- a function
OUTPUT: a pair ``(result, suggestion)``
``result`` is the sorted list obtained by merging the lists in ``lists`` while removing duplicates, and ``suggestion`` is a list such that applying ``C3`` on ``lists`` with its last list replaced by ``suggestion`` would return ``result``.
EXAMPLES:
With the following input, :func:`C3_merge` returns right away a sorted list::
sage: from sage.misc.c3_controlled import C3_merge sage: C3_merge([[2],[1]]) [2, 1]
In that case, :func:`C3_sorted_merge` returns the same result, with the last line unchanged::
sage: from sage.misc.c3_controlled import C3_sorted_merge sage: C3_sorted_merge([[2],[1]]) ([2, 1], [1])
On the other hand, with the following input, :func:`C3_merge` returns a non sorted list::
sage: C3_merge([[1],[2]]) [1, 2]
Then, :func:`C3_sorted_merge` returns a sorted list, and suggests to replace the last line by ``[2,1]``::
sage: C3_sorted_merge([[1],[2]]) ([2, 1], [2, 1])
And indeed ``C3_merge`` now returns the desired result::
sage: C3_merge([[1],[2,1]]) [2, 1]
From now on, we use this little wrapper that checks that ``C3_merge``, with the suggestion of ``C3_sorted_merge``, returns a sorted list::
sage: def C3_sorted_merge_check(lists): ....: result, suggestion = C3_sorted_merge(lists) ....: assert result == C3_merge(lists[:-1] + [suggestion]) ....: return result, suggestion
Base cases::
sage: C3_sorted_merge_check([]) Traceback (most recent call last): ... ValueError: The input should be a non empty list of lists (or iterables) sage: C3_sorted_merge_check([[]]) ([], []) sage: C3_sorted_merge_check([[1]]) ([1], [1]) sage: C3_sorted_merge_check([[3,2,1]]) ([3, 2, 1], [3, 2, 1]) sage: C3_sorted_merge_check([[1],[1]]) ([1], [1]) sage: C3_sorted_merge_check([[3,2,1],[3,2,1]]) ([3, 2, 1], [3, 2, 1])
Exercise different states for the last line::
sage: C3_sorted_merge_check([[1],[2],[]]) ([2, 1], [2, 1]) sage: C3_sorted_merge_check([[1],[2], [1]]) ([2, 1], [2, 1])
Explore (all?) the different execution branches::
sage: C3_sorted_merge_check([[3,1],[4,2]]) ([4, 3, 2, 1], [4, 3, 2, 1]) sage: C3_sorted_merge_check([[4,1],[3,2]]) ([4, 3, 2, 1], [3, 2, 1]) sage: C3_sorted_merge_check([[3,2],[4,1]]) ([4, 3, 2, 1], [4, 3, 1]) sage: C3_sorted_merge_check([[1],[4,3,2]]) ([4, 3, 2, 1], [4, 3, 2, 1]) sage: C3_sorted_merge_check([[1],[3,2], []]) ([3, 2, 1], [2, 1]) sage: C3_sorted_merge_check([[1],[4,3,2], []]) ([4, 3, 2, 1], [2, 1]) sage: C3_sorted_merge_check([[1],[4,3,2], [2]]) ([4, 3, 2, 1], [2, 1]) sage: C3_sorted_merge_check([[2],[1],[4],[3]]) ([4, 3, 2, 1], [3, 2, 1]) sage: C3_sorted_merge_check([[2],[1],[4],[]]) ([4, 2, 1], [4, 2, 1]) sage: C3_sorted_merge_check([[2],[1],[3],[4]]) ([4, 3, 2, 1], [4, 3, 2, 1]) sage: C3_sorted_merge_check([[2],[1],[3,2,1],[3]]) ([3, 2, 1], [3]) sage: C3_sorted_merge_check([[2],[1],[2,1],[3]]) ([3, 2, 1], [3, 2])
Exercises adding one item when the last list has a single element; the second example comes from an actual poset::
sage: C3_sorted_merge_check([[5,4,2],[4,3],[5,4,1]]) ([5, 4, 3, 2, 1], [5, 4, 3, 2, 1]) sage: C3_sorted_merge_check([[6,4,2],[5,3],[6,5,1]]) ([6, 5, 4, 3, 2, 1], [6, 5, 4, 3, 2, 1]) """ #for l in lists: # assert sorted(l, key = key, reverse=True) == l,\ # "Each input list should be sorted %s"%l
# Data structure / invariants: # - Each list remains sorted and duplicate free. # - Each list only evolves by popping its largest element # Exception: elements can be inserted back into the last list. # - Whenever a list becomes empty, it's removed from the data structure. # The order between the (remaining non empty) lists remains unchanged. # - nbheads contains the number of lists appearing in the data structure. # - The flag ``last_list_non_empty`` states whether the last # list is currently non empty; if yes, by the above, this list is stored last. # - Each list is split between its head (in ``heads``) and tail (in ``tails''). # - Each tail is stored reversed, so that we can use a cheap ``pop()`` # in lieue of ``pop(0)``. # - A duplicate of this tail is stored as a set (of keys) in # ``tailsets``, for cheap membership testing.
cdef int i, j, max_i cdef bint cont cdef list tail, l cdef set tailset
cdef set tmp_set # for i in range(len(tails)): # assert len(tails[i]) == len(tailsets[i]), \ # "All objects should be distinct and have distinct sorting key!"+'\n'.join(" - %s: %s"%(key(O), O) for O in sorted(tails[i], key=key))
# def print_state(): # print("-- %s -- %s ------"%(out,suggestion)) # for i in range(nbheads): # print([heads[i]] + list(reversed(tails[i])))
# def check_state(): # for i in range(nbheads): # l = tails[i] # if heads[i] is not None: # l = l+[heads[i]] # assert sorted(l, key=key) == l # assert len(set(l)) == len(l) # assert len(tails[i]) == len(set(tails[i])), \ # "C3's input list should have no repeats %s"%tails[i] # assert set(key(O) for O in tails[i]) == tailsets[i], \ # "inconsistent tails[i] and tailsets[i]: %s %s"%(tails[i], tailsets[i]) # assert len(tails[i]) == len(tailsets[i]), \ # "keys should be distinct"%(tails[i])
#print_state() #check_state() # Find the position of the largest head which will become the next item
# Find all the bad choices # Does O appear in none of the tails? # replace the closure # if any(O_key in tailsets[j] for j in range(nbheads) if j != i): continue
# The plain C3 algorithm would have chosen O as next item!
# We prevent this choice by inserting O in the tail of the # suggestions. At this stage, we only insert it into the # last list. Later, we will make sure that it is actually # in the tail of the last list. # Reinstate the last list for the suggestion # if it had disappeared before # Use a heap or something for fast sorted insertion? # Since Python uses TimSort, that's probably not so bad. #check_state()
# Insert max_value in the last list, if needed to hold off the bad items tails[-1].append(last_head) tailsets[-1].add(key(last_head)) #check_state() #check_state()
# Clear O from other heads, removing the line altogether # if the tail is already empty. # j goes down so that ``del heads[j]`` does not screw up the numbering else: #check_state() #assert C3_merge(lists[:-1]+[suggestion_list]) == out
class HierarchyElement(object, metaclass=ClasscallMetaclass): """ A class for elements in a hierarchy.
This class is for testing and experimenting with various variants of the ``C3`` algorithm to compute a linear extension of the elements above an element in a hierarchy. Given the topic at hand, we use the following naming conventions. For `x` an element of the hierarchy, we call the elements just above `x` its *bases*, and the linear extension of all elements above `x` its *MRO*.
By convention, the bases are given as lists of ``HierarchyElement`` s, and MROs are given a list of the corresponding values.
INPUT:
- ``value`` -- an object - ``succ`` -- a successor function, poset or digraph from which one can recover the successors of ``value`` - ``key`` -- a function taking values as input (default: the identity) this function is used to compute comparison keys for sorting elements of the hierarchy.
.. NOTE::
Constructing a ``HierarchyElement`` immediately constructs the whole hierarchy above it.
EXAMPLES:
See the introduction of this module :mod:`sage.misc.c3_controlled` for many examples. Here we consider a large example, originaly taken from the hierarchy of categories above :class:`HopfAlgebrasWithBasis`::
sage: from sage.misc.c3_controlled import HierarchyElement sage: G = DiGraph({ ....: 44 : [43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 43 : [42, 41, 40, 36, 35, 39, 38, 37, 33, 32, 31, 30, 29, 28, 27, 26, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 42 : [36, 35, 37, 30, 29, 28, 27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 41 : [40, 36, 35, 33, 32, 31, 30, 29, 28, 27, 26, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 40 : [36, 35, 32, 31, 30, 29, 28, 27, 26, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 39 : [38, 37, 33, 32, 31, 30, 29, 28, 27, 26, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 38 : [37, 33, 32, 31, 30, 29, 28, 27, 26, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 37 : [30, 29, 28, 27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 36 : [35, 30, 29, 28, 27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 35 : [29, 28, 27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 34 : [33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 33 : [32, 31, 30, 29, 28, 27, 26, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 32 : [31, 30, 29, 28, 27, 26, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 31 : [30, 29, 28, 27, 26, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 30 : [29, 28, 27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 29 : [28, 27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 28 : [27, 26, 15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 27 : [15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 26 : [15, 14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 25 : [24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 24 : [4, 2, 1, 0], ....: 23 : [22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 22 : [21, 20, 18, 17, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 21 : [20, 17, 4, 2, 1, 0], ....: 20 : [4, 2, 1, 0], ....: 19 : [18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 18 : [17, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 17 : [4, 2, 1, 0], ....: 16 : [15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 15 : [14, 12, 11, 9, 8, 5, 3, 2, 1, 0], ....: 14 : [11, 3, 2, 1, 0], ....: 13 : [12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 12 : [11, 9, 8, 5, 3, 2, 1, 0], ....: 11 : [3, 2, 1, 0], ....: 10 : [9, 8, 7, 6, 5, 4, 3, 2, 1, 0], ....: 9 : [8, 5, 3, 2, 1, 0], ....: 8 : [3, 2, 1, 0], ....: 7 : [6, 5, 4, 3, 2, 1, 0], ....: 6 : [4, 3, 2, 1, 0], ....: 5 : [3, 2, 1, 0], ....: 4 : [2, 1, 0], ....: 3 : [2, 1, 0], ....: 2 : [1, 0], ....: 1 : [0], ....: 0 : [], ....: })
sage: x = HierarchyElement(44, G) sage: x.mro [44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0] sage: x.cls <class '44.cls'> sage: x.cls.mro() [<class '44.cls'>, <class '43.cls'>, <class '42.cls'>, <class '41.cls'>, <class '40.cls'>, <class '39.cls'>, <class '38.cls'>, <class '37.cls'>, <class '36.cls'>, <class '35.cls'>, <class '34.cls'>, <class '33.cls'>, <class '32.cls'>, <class '31.cls'>, <class '30.cls'>, <class '29.cls'>, <class '28.cls'>, <class '27.cls'>, <class '26.cls'>, <class '25.cls'>, <class '24.cls'>, <class '23.cls'>, <class '22.cls'>, <class '21.cls'>, <class '20.cls'>, <class '19.cls'>, <class '18.cls'>, <class '17.cls'>, <class '16.cls'>, <class '15.cls'>, <class '14.cls'>, <class '13.cls'>, <class '12.cls'>, <class '11.cls'>, <class '10.cls'>, <class '9.cls'>, <class '8.cls'>, <class '7.cls'>, <class '6.cls'>, <class '5.cls'>, <class '4.cls'>, <class '3.cls'>, <class '2.cls'>, <class '1.cls'>, <class '0.cls'>, <... 'object'>] """ @staticmethod def __classcall__(cls, value, succ, key = None): """ EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: x = HierarchyElement(10, P) sage: x 10 sage: x.bases [5, 2] sage: x.mro [10, 5, 2, 1] """
def __init__(self, value, bases, key, from_value): """ EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: x = HierarchyElement(10, P) sage: x 10 sage: x.value 10 sage: x._bases [5, 2] sage: x._key <built-in function identity> sage: x._key(10) 10
The ``_from_value`` attribute is a function that can be used to reconstruct an element of the hierarchy from its value::
sage: x._from_value Cached version of <cyfunction HierarchyElement.__classcall__.<locals>.f at ...> sage: x._from_value(x.value) is x True """
def __repr__(self): """ Return the representation of ``self`` which is that of its value.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: x = HierarchyElement(10, P) sage: x 10 """
@lazy_attribute def bases(self): """ The bases of ``self``.
The bases are given as a list of ``HierarchyElement``s, sorted decreasingly accoding to the ``key`` function.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: x = HierarchyElement(10, P) sage: x.bases [5, 2] sage: type(x.bases[0]) <class 'sage.misc.c3_controlled.HierarchyElement'> sage: x.mro [10, 5, 2, 1] sage: x._bases_controlled [5, 2] """
@lazy_attribute def mro(self): """ The MRO for this object, calculated with :meth:`C3_sorted_merge`.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement, C3_sorted_merge, identity sage: P = Poset({7: [5,6], 5:[1,2], 6: [3,4]}, facade = True) sage: x = HierarchyElement(5, P) sage: x.mro [5, 2, 1] sage: x = HierarchyElement(6, P) sage: x.mro [6, 4, 3] sage: x = HierarchyElement(7, P) sage: x.mro [7, 6, 5, 4, 3, 2, 1]
sage: C3_sorted_merge([[6, 4, 3], [5, 2, 1], [6, 5]], identity) ([6, 5, 4, 3, 2, 1], [6, 5, 4])
TESTS::
sage: assert all(isinstance(v, Integer) for v in x.mro) """
@lazy_attribute def _bases_controlled(self): """ A list of bases controlled by :meth:`C3_sorted_merge`
This triggers the calculation of the MRO using :meth:`C3_sorted_merge`, which sets this attribute as a side effect.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset({7: [5,6], 5:[1,2], 6: [3,4]}, facade = True) sage: x = HierarchyElement(7, P) sage: x._bases [6, 5] sage: x._bases_controlled [6, 5, 4] """
@lazy_attribute def mro_standard(self): """ The MRO for this object, calculated with :meth:`C3_merge`
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement, C3_merge sage: P = Poset({7: [5,6], 5:[1,2], 6: [3,4]}, facade=True) sage: x = HierarchyElement(5, P) sage: x.mro_standard [5, 2, 1] sage: x = HierarchyElement(6, P) sage: x.mro_standard [6, 4, 3] sage: x = HierarchyElement(7, P) sage: x.mro_standard [7, 6, 4, 3, 5, 2, 1] sage: C3_merge([[6, 4, 3], [5, 2, 1], [6, 5]]) [6, 4, 3, 5, 2, 1]
TESTS::
sage: assert all(isinstance(v, Integer) for v in x.mro_standard) """
@lazy_attribute def mro_controlled(self): """ The MRO for this object, calculated with :meth:`C3_merge`, under control of `C3_sorted_merge`
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement, C3_merge sage: P = Poset({7: [5,6], 5:[1,2], 6: [3,4]}, facade=True) sage: x = HierarchyElement(5, P) sage: x.mro_controlled [5, 2, 1] sage: x = HierarchyElement(6, P) sage: x.mro_controlled [6, 4, 3] sage: x = HierarchyElement(7, P) sage: x.mro_controlled [7, 6, 5, 4, 3, 2, 1] sage: x._bases [6, 5] sage: x._bases_controlled [6, 5, 4] sage: C3_merge([[6, 4, 3], [5, 2, 1], [6, 5]]) [6, 4, 3, 5, 2, 1] sage: C3_merge([[6, 4, 3], [5, 2, 1], [6, 5, 4]]) [6, 5, 4, 3, 2, 1]
TESTS::
sage: assert all(isinstance(v, Integer) for v in x.mro_controlled) """
@cached_method def _test_mro(self): r""" Runs consistency tests.
This checks in particular that the instrumented ``C3`` and controlled ``C3`` algorithms give, as desired, the decreasingly sorted list of the objects above in the hierarchy. For the controlled ``C3`` algorithm, this includes both Sage's implementation, and Python's implementation (by constructing an appropriate hierarchy of classes).
It is cached because it is run recursively on the elements above ``self``.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset({7: [5,6], 5:[1,2], 6: [3,4]}, facade=True) sage: x = HierarchyElement(7, P) sage: x._test_mro() """ # standard C3 failed to compute a mro; that's ok pass
@lazy_attribute def cls(self): """ Return a Python class with inheritance graph parallel to the hierarchy above ``self``.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: x = HierarchyElement(1, P) sage: x.cls <class '1.cls'> sage: x.cls.mro() [<class '1.cls'>, <... 'object'>] sage: x = HierarchyElement(30, P) sage: x.cls <class '30.cls'> sage: x.cls.mro() [<class '30.cls'>, <class '15.cls'>, <class '10.cls'>, <class '6.cls'>, <class '5.cls'>, <class '3.cls'>, <class '2.cls'>, <class '1.cls'>, <... 'object'>] """
@cached_method def all_bases(self): """ Return the set of all the ``HierarchyElement``s above ``self``, ``self`` included.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: HierarchyElement(1, P).all_bases() {1} sage: HierarchyElement(10, P).all_bases() # random output {10, 5, 2, 1} sage: sorted([x.value for x in HierarchyElement(10, P).all_bases()]) [1, 2, 5, 10] """
def all_bases_len(self): """ Return the cumulated size of the bases of the elements above ``self`` in the hierarchy.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: HierarchyElement(30, P).all_bases_len() 12 """
def all_bases_controlled_len(self): """ Return the cumulated size of the controlled bases of the elements above ``self`` in the hierarchy.
EXAMPLES::
sage: from sage.misc.c3_controlled import HierarchyElement sage: P = Poset((divisors(30), lambda x,y: y.divides(x)), facade=True) sage: HierarchyElement(30, P).all_bases_controlled_len() 13 """ |