psycodict.encoding

This module provides functions for encoding data for storage in Postgres and decoding the results.

psycodict.encoding.numeric_precision(value)

The bit precision needed to faithfully represent a decimal string: log(10)/log(2) bits per significant digit, where the sign, the decimal point and leading zeros carry no information. (Counting every character, as this function’s predecessor did, manufactured phantom digits whenever the value was printed at full precision.) At least 2, the smallest precision a RealField accepts.

INPUT:

• value – a string representing a decimal number, as Postgres delivers the numeric type

EXAMPLES:

>>> numeric_precision("0.00459244230167")  # 12 significant digits
40

psycodict.encoding.numeric_converter(value, cur=None)

Used for converting numeric values from Postgres to Python.

INPUT:

• value – a string representing a decimal number.

• cur – a cursor, unused

OUTPUT:

• either a sage integer (if there is no decimal point) or a real number whose precision depends on the number of significant digits in value.

class psycodict.encoding.Array(seq)

Bases: object

Since we use Json by default for lists, this class lets us get back the original behavior of encoding as a Postgres array when needed.

getquoted()

The Postgres array literal for the wrapped sequence, as bytes.

class psycodict.encoding.ArrayDumper(cls: type, context: AdaptContext | None = None)

Bases: Dumper

Dumps an Array wrapper as a Postgres array literal with unknown oid.

dump(obj)

Render the wrapped sequence as a Postgres array literal.

psycodict.encoding.homogenize_int_arrays(value)

Return value with the entries of an all-integer list made into Python ints, if they are not all of one type already.

psycopg dumps a list by picking one dumper for the whole array, so it refuses a list whose entries have different types unless those types dump to the same Postgres oid: [Integer(1), 2], mixing Sage’s integers with Python’s, raises DataError: cannot dump lists of mixed types. Such a list means one thing only – an array of integers – and callers assembling one out of several sources (a parsed search box here, a literal there) have no reason to care which flavour of integer each source produced, so psycodict makes them agree rather than making the query fail.

Only lists that mix an integer type psycopg does not know natively into an otherwise integral list are touched, so this changes the behaviour of no query that worked before:

• a list of a single type dumps fine as it is, whatever that type is;

• a list holding anything non-integral is left for psycopg to judge;

• bool is integral to Python but not to Postgres, so a list holding one is left alone;

• subclasses of int (which is how psycopg’s own Int2/Int4/ Int8/IntNumeric wrappers ask for a particular oid) are left alone, since replacing them would discard the oid they were chosen for.

INPUT:

• value – anything that might be passed to Postgres as a value

OUTPUT:

Either value itself, or a list of ints with the same shape.

EXAMPLES:

>>> import numbers
>>> from psycodict.encoding import homogenize_int_arrays

Sage’s Integer is the type this exists for. psycodict does not require Sage, so the examples below use a stand-in with the one property that matters here, being registered as an integer:

>>> class MyInt:
...     def __init__(self, n):
...         self.n = n
...     def __int__(self):
...         return self.n
>>> _ = numbers.Integral.register(MyInt)

A list of Python ints is homogeneous already and comes back unchanged, as does anything that is not a list at all:

>>> homogenize_int_arrays([1, -1, 1])
[1, -1, 1]
>>> homogenize_int_arrays(3)
3

Mixing in the other integer type would make psycopg refuse the array, so the entries become ints:

>>> homogenize_int_arrays([MyInt(1), -1, 1])
[1, -1, 1]

A nested array is one array, as it is in Postgres, and its NULLs stay NULL:

>>> homogenize_int_arrays([[MyInt(1), 2], [3, None]])
[[1, 2], [3, None]]

A list that is not all integers is psycopg’s business, not ours:

>>> homogenize_int_arrays([0.5, 1])
[0.5, 1]

psycodict.encoding.homogenize_values(values, values_list=False)

Return the values of a query with every mixed-type integer array in them made homogeneous by homogenize_int_arrays().

INPUT:

• values – the values of a single statement: a sequence with one entry per %s placeholder, or a mapping keyed by the names of %(name)s ones. When values_list is set, a sequence of such collections instead, one per row of a multi-row insert.

• values_list – boolean (default False); whether values is a sequence of rows rather than the values of one statement

OUTPUT:

values itself when nothing needed changing, which is the usual case, so that ordinary queries copy nothing.

EXAMPLES:

>>> from psycodict.encoding import homogenize_values

A query whose values hold no mixed array – almost every query – gets its own values back, not a copy of them:

>>> values = [[1, 2], 'a', 3]
>>> homogenize_values(values) is values
True
>>> rows = [[[1, 2], 'a'], [[3, 4], 'b']]
>>> homogenize_values(rows, values_list=True) is rows
True

class psycodict.encoding.Json(obj)

Bases: object

A wrapper marking a value for storage as json/jsonb, encoded with psycodict’s extended encoding (Sage types etc.).

With psycopg2 this subclassed psycopg2.extras.Json; with psycopg3 adaptation happens through JsonWrapperDumper below instead. The wrapped value is available as both .obj (psycopg3 convention) and .adapted (psycopg2 convention, kept for backward compatibility).

classmethod dumps(obj)

Serialize obj to json text using the extended encoding (see prep()).

classmethod loads(s)

Parse json text and decode the extended encoding back to Python and Sage objects (see extract()).

classmethod prep(obj, escape_backslashes=False)

Returns a version of the object that is parsable by the standard json dumps function. For example, replace Integers with ints, encode various Sage types using dictionaries….

classmethod extract(obj)

Takes an object extracted by the json parser and decodes the special-formating dictionaries used to store special types.

class psycodict.encoding.JsonWrapperDumper(cls: type, context: AdaptContext | None = None)

Bases: Dumper

Dumps a Json wrapper as its json text. We leave the oid unknown (0) rather than declaring json/jsonb, matching psycopg2’s behavior of interpolating an untyped quoted literal so that the server casts by context (this works for both json and jsonb columns).

dump(obj)

Render the wrapped value as json text.

class psycodict.encoding.DictJsonDumper(cls: type, context: AdaptContext | None = None)

Bases: Dumper

Dumps a plain dict as json text with unknown oid (psycopg2 behavior via register_adapter(dict, Json)).

dump(obj)

Render the dict as json text.

psycodict.encoding.check_copy_sep(sep)

Raise a ValueError if sep cannot be used as the column separator for a COPY (text format) file written by copy_dumps().

psycodict.encoding.copy_dumps(inp, typ, recursing=False, *, sep='|')

Output a string formatted as needed for loading by Postgres’ COPY FROM.

INPUT:

• inp – a Python or Sage object that directly translates to a postgres type (e.g. Integer, RealLiteral, dict…

• typ – the Postgres type of the column in which this data is being stored.

• recursing – used internally to format the elements of an array. The value is rendered as an array member (quoted and escaped for the array parser as needed) rather than as a complete COPY field, and sep plays no role: the separator is escaped in a single pass over the assembled field by the top-level call.

• sep – (keyword only) the column separator that the resulting file will be loaded with (default "|"). Occurrences of this character anywhere in the formatted field – inside text values, but also in dates and negative numbers for sep="-", times for sep=":", JSON text and the braces, commas and quotes of array literals – are backslash-escaped, matching what Postgres’ COPY (text format) does on output, so that they are not mistaken for column boundaries on the way back in. This must agree with the separator eventually passed to COPY FROM. Separators that cannot work (see check_copy_sep()) raise a ValueError.