# Schema and converter reference

All names below are exported from `fsspec_data`.

## Schema values

| Type               | Fields                                                                     |
| ------------------ | -------------------------------------------------------------------------- |
| `SchemaRef`        | `url`, optional `format`, `storage_options` (default empty mapping).       |
| `SchemaProvenance` | `source`, `format`, `provider`, optional `identifier`, optional `version`. |
| `ResolvedSchema`   | `schema: pyarrow.Schema`, `provenance: SchemaProvenance`.                  |
| `SchemaDocument`   | `data: bytes`, `format: SchemaFormat`, `provenance: SchemaProvenance`.     |

`SchemaRef` copies storage options and omits them from its representation.
`SchemaFormat` values are `arrow-json`, `arrow-ipc-schema`, and `json-schema`.
`.arrow`, `.ipc`, and `.arrowschema` infer `arrow-ipc-schema`; other suffixes require
an explicit format.

`arrow-json` is an object with a `fields` list. Each field has `name`, `type`, and
optional `nullable` (default true). Types accept PyArrow aliases.
`arrow-ipc-schema` accepts a serialized Arrow schema or Arrow IPC stream.

### `resolve_schema(value, registry=None)`

Accepts a PyArrow schema, `SchemaRef`, or `None`. Returns `ResolvedSchema` or
`None`. Inline Arrow schemas have `source="inline"`, `format="arrow"`, and
`provider="inline"`. References use `registry` or `DEFAULT_SCHEMA_RESOLVERS`.
The public function expects an already constructed reference; `DataFileSystem`
also accepts URL strings and reference mappings and normalizes them first.

### `SchemaResolverRegistry`

`register(format, decoder)` installs a `bytes -> pyarrow.Schema` decoder.
`register_provider(protocol, provider)` installs a `SchemaRef -> SchemaDocument`
provider. `resolve(reference)` invokes the provider and decoder and returns
`ResolvedSchema`. Unregistered protocols use fsspec document loading. A reference
format that disagrees with the provider's document format raises `ValueError`.

### `json_schema_to_arrow(document)`

Translates a structural JSON Schema rooted at an object. Objects, homogeneous
arrays, strings, booleans, integers, numbers, and nulls are supported. Unions may
contain null and one other type. Required properties control field nullability;
optional properties and explicit null unions are nullable.

References, composition keywords, multiple non-null union members, schema-valued
`additionalProperties`, tuple arrays, and `format` annotations raise `ValueError`
with the unsupported field path. Validation-only constraints such as numeric
bounds and string lengths are not represented or enforced.

### Confluent provider

`confluent://SUBJECT/versions/VERSION` accepts `latest`, `-1`, or a positive 32-bit
integer. Its storage options are required `registry_url`, optional Basic auth
`username` and `password`, optional string-valued `headers`, and optional numeric
`timeout` in seconds. JSON Schema records without external references are
supported; Avro, Protobuf, and referenced documents raise `ValueError`.

## Resource converters

### `Converter(source_type, target_type, handler)`

Frozen route descriptor. Route names are stripped and lowercased. Empty or
non-string names raise `ValueError`; a non-callable handler raises `TypeError`.

`convert(source, target=None, *, source_options=None, target_options=None, conversion_options=None)` calls the handler with independent copied option
mappings. Invalid option mappings raise `TypeError`. Return values are handler-defined.

### `ConverterRegistry(*, entry_point_group="fsspec_data.converters", discover_entry_points=True)`

| Method                                                              | Behavior                                                                                                                                    |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `register(converter, replace=False)`                                | Registers a `Converter`; duplicate routes raise `ValueError` unless replacement is requested.                                               |
| `load_entry_points()`                                               | Discovers descriptors once, unless discovery is disabled. Non-converter descriptors raise `TypeError`; duplicate routes raise `ValueError`. |
| `get(source_type, target_type)`                                     | Loads entry points and returns a route; missing routes raise `ValueError`.                                                                  |
| `convert(source_type, target_type, source, target=None, **options)` | Invokes the selected converter with the three option mappings.                                                                              |

`CONVERTER_ENTRY_POINT_GROUP` is `fsspec_data.converters`.
`DEFAULT_CONVERTERS` includes `zarr -> xarray` and `xarray -> zarr`; dependencies
are installed with the `xarray` extra.

`zarr -> xarray` returns `xarray.open_zarr(source, **conversion_options)` and
accepts source storage options. It rejects a target and target options.
`xarray -> zarr` requires a Dataset or DataArray, calls `to_zarr`, and accepts
target storage options. It rejects source options. Storage options supplied both
through their dedicated mapping and `conversion_options` raise `ValueError`.