Me parece que Pandas ExtensionArray s sería uno de los casos en los que un ejemplo simple para comenzar realmente ayudaría. Sin embargo, no he encontrado un ejemplo lo suficientemente simple en ninguna parte.
ExtensionArray Para crear un ExtensionArray , necesita
ExtensionDtype y regístreloExtensionArray implementando los métodos requeridos.También hay una sección en la documentación de Pandas con una breve descripción general.
Hay muchos ejemplos de implementaciones:
GeometryArray de GeopandasStringSupportingExtensionArray de FletcherA pesar de haber estudiado todo lo anterior, todavía encuentro que las matrices de extensión son difíciles de entender. Todos los ejemplos tienen muchos detalles y funciones personalizadas que dificultan determinar qué es realmente necesario. Sospecho que muchos se han enfrentado a un problema similar.
Por lo tanto, estoy pidiendo un ejemplo simple y mínimo de un ExtensionArray que funcione. La clase debe pasar todas las pruebas proporcionadas por Pandas para verificar que ExtensionArray se comporte como se esperaba. He proporcionado un ejemplo de implementación de las pruebas a continuación.
Para tener un ejemplo concreto, digamos que quiero extender ExtensionArray para obtener una matriz de enteros que pueda contener valores NA. Eso es esencialmente IntegerArray , pero despojado de cualquier funcionalidad real más allá de los conceptos básicos de ExtensionArray .
He utilizado los siguientes accesorios y pruebas para probar la validez de la solución. Estos se basan en las instrucciones de la documentación de Pandas.
import operator import numpy as np from pandas import Series import pytest from pandas.tests.extension.base.casting import BaseCastingTests # noqa from pandas.tests.extension.base.constructors import BaseConstructorsTests # noqa from pandas.tests.extension.base.dtype import BaseDtypeTests # noqa from pandas.tests.extension.base.getitem import BaseGetitemTests # noqa from pandas.tests.extension.base.groupby import BaseGroupbyTests # noqa from pandas.tests.extension.base.interface import BaseInterfaceTests # noqa from pandas.tests.extension.base.io import BaseParsingTests # noqa from pandas.tests.extension.base.methods import BaseMethodsTests # noqa from pandas.tests.extension.base.missing import BaseMissingTests # noqa from pandas.tests.extension.base.ops import ( # noqa BaseArithmeticOpsTests, BaseComparisonOpsTests, BaseOpsUtil, BaseUnaryOpsTests, ) from pandas.tests.extension.base.printing import BasePrintingTests # noqa from pandas.tests.extension.base.reduce import ( # noqa BaseBooleanReduceTests, BaseNoReduceTests, BaseNumericReduceTests, ) from pandas.tests.extension.base.reshaping import BaseReshapingTests # noqa from pandas.tests.extension.base.setitem import BaseSetitemTests # noqa from .extension import NullableIntArray @pytest.fixture def dtype(): """A fixture providing the ExtensionDtype to validate.""" return 'NullableInt' @pytest.fixture def data(): """ Length-100 array for this type. * data[0] and data[1] should both be non missing * data[0] and data[1] should not be equal """ return NullableIntArray(np.array(list(range(100)))) @pytest.fixture def data_for_twos(): """Length-100 array in which all the elements are two.""" return NullableIntArray(np.array([2] * 2)) @pytest.fixture def data_missing(): """Length-2 array with [NA, Valid]""" return NullableIntArray(np.array([np.nan, 2])) @pytest.fixture(params=["data", "data_missing"]) def all_data(request, data, data_missing): """Parametrized fixture giving 'data' and 'data_missing'""" if request.param == "data": return data elif request.param == "data_missing": return data_missing @pytest.fixture def data_repeated(data): """ Generate many datasets. Parameters ---------- data : fixture implementing `data` Returns ------- Callable[[int], Generator]: A callable that takes a `count` argument and returns a generator yielding `count` datasets. """ def gen(count): for _ in range(count): yield data return gen @pytest.fixture def data_for_sorting(): """ Length-3 array with a known sort order. This should be three items [B, C, A] with A < B < C """ return NullableIntArray(np.array([2, 3, 1])) @pytest.fixture def data_missing_for_sorting(): """ Length-3 array with a known sort order. This should be three items [B, NA, A] with A < B and NA missing. """ return NullableIntArray(np.array([2, np.nan, 1])) @pytest.fixture def na_cmp(): """ Binary operator for comparing NA values. Should return a function of two arguments that returns True if both arguments are (scalar) NA for your type. By default, uses ``operator.is_`` """ return operator.is_ @pytest.fixture def na_value(): """The scalar missing value for this type. Default 'None'""" return np.nan @pytest.fixture def data_for_grouping(): """ Data for factorization, grouping, and unique tests. Expected to be like [B, B, NA, NA, A, A, B, C] Where A < B < C and NA is missing """ return NullableIntArray(np.array([2, 2, np.nan, np.nan, 1, 1, 2, 3])) @pytest.fixture(params=[True, False]) def box_in_series(request): """Whether to box the data in a Series""" return request.param @pytest.fixture( params=[ lambda x: 1, lambda x: [1] * len(x), lambda x: Series([1] * len(x)), lambda x: x, ], ids=["scalar", "list", "series", "object"], ) def groupby_apply_op(request): """ Functions to test groupby.apply(). """ return request.param @pytest.fixture(params=[True, False]) def as_frame(request): """ Boolean fixture to support Series and Series.to_frame() comparison testing. """ return request.param @pytest.fixture(params=[True, False]) def as_series(request): """ Boolean fixture to support arr and Series(arr) comparison testing. """ return request.param @pytest.fixture(params=[True, False]) def use_numpy(request): """ Boolean fixture to support comparison testing of ExtensionDtype array and numpy array. """ return request.param @pytest.fixture(params=["ffill", "bfill"]) def fillna_method(request): """ Parametrized fixture giving method parameters 'ffill' and 'bfill' for Series.fillna(method=<method>) testing. """ return request.param @pytest.fixture(params=[True, False]) def as_array(request): """ Boolean fixture to support ExtensionDtype _from_sequence method testing. """ return request.param class TestCastingTests(BaseCastingTests): pass class TestConstructorsTests(BaseConstructorsTests): pass class TestDtypeTests(BaseDtypeTests): pass class TestGetitemTests(BaseGetitemTests): pass class TestGroupbyTests(BaseGroupbyTests): pass class TestInterfaceTests(BaseInterfaceTests): pass class TestParsingTests(BaseParsingTests): pass class TestMethodsTests(BaseMethodsTests): pass class TestMissingTests(BaseMissingTests): pass class TestArithmeticOpsTests(BaseArithmeticOpsTests): pass class TestComparisonOpsTests(BaseComparisonOpsTests): pass class TestOpsUtil(BaseOpsUtil): pass class TestUnaryOpsTests(BaseUnaryOpsTests): pass class TestPrintingTests(BasePrintingTests): pass class TestBooleanReduceTests(BaseBooleanReduceTests): pass class TestNoReduceTests(BaseNoReduceTests): pass class TestNumericReduceTests(BaseNumericReduceTests): pass class TestReshapingTests(BaseReshapingTests): pass class TestSetitemTests(BaseSetitemTests): pass