The model reference documentation explains how to use
Django’s standard field classes – CharField
,
DateField
, etc. For many purposes, those classes are
all you’ll need. Sometimes, though, the Django version won’t meet your precise
requirements, or you’ll want to use a field that is entirely different from
those shipped with Django.
Django’s built-in field types don’t cover every possible database column type –
only the common types, such as VARCHAR
and INTEGER
. For more obscure
column types, such as geographic polygons or even user-created types such as
PostgreSQL custom types, you can define your own Django Field
subclasses.
Alternatively, you may have a complex Python object that can somehow be
serialized to fit into a standard database column type. This is another case
where a Field
subclass will help you use your object with your models.
Creating custom fields requires a bit of attention to detail. To make things easier to follow, we’ll use a consistent example throughout this document: wrapping a Python object representing the deal of cards in a hand of Bridge. Don’t worry, you don’t have know how to play Bridge to follow this example. You only need to know that 52 cards are dealt out equally to four players, who are traditionally called north, east, south and west. Our class looks something like this:
class Hand(object):
"""A hand of cards (bridge style)"""
def __init__(self, north, east, south, west):
# Input parameters are lists of cards ('Ah', '9s', etc)
self.north = north
self.east = east
self.south = south
self.west = west
# ... (other possibly useful methods omitted) ...
This is just an ordinary Python class, with nothing Django-specific about it.
We’d like to be able to do things like this in our models (we assume the
hand
attribute on the model is an instance of Hand
):
example = MyModel.objects.get(pk=1)
print example.hand.north
new_hand = Hand(north, east, south, west)
example.hand = new_hand
example.save()
We assign to and retrieve from the hand
attribute in our model just like
any other Python class. The trick is to tell Django how to handle saving and
loading such an object.
In order to use the Hand
class in our models, we do not have to change
this class at all. This is ideal, because it means you can easily write
model support for existing classes where you cannot change the source code.
Note
You might only be wanting to take advantage of custom database column
types and deal with the data as standard Python types in your models;
strings, or floats, for example. This case is similar to our Hand
example and we’ll note any differences as we go along.
The simplest way to think of a model field is that it provides a way to take a
normal Python object – string, boolean, datetime
, or something more
complex like Hand
– and convert it to and from a format that is useful
when dealing with the database (and serialization, but, as we’ll see later,
that falls out fairly naturally once you have the database side under control).
Fields in a model must somehow be converted to fit into an existing database column type. Different databases provide different sets of valid column types, but the rule is still the same: those are the only types you have to work with. Anything you want to store in the database must fit into one of those types.
Normally, you’re either writing a Django field to match a particular database column type, or there’s a fairly straightforward way to convert your data to, say, a string.
For our Hand
example, we could convert the card data to a string of 104
characters by concatenating all the cards together in a pre-determined order –
say, all the north cards first, then the east, south and west cards. So
Hand
objects can be saved to text or character columns in the database.
Field
¶All of Django’s fields (and when we say fields in this document, we always
mean model fields and not form fields) are subclasses
of django.db.models.Field
. Most of the information that Django records
about a field is common to all fields – name, help text, uniqueness and so
forth. Storing all that information is handled by Field
. We’ll get into the
precise details of what Field
can do later on; for now, suffice it to say
that everything descends from Field
and then customizes key pieces of the
class behavior.
It’s important to realize that a Django field class is not what is stored in
your model attributes. The model attributes contain normal Python objects. The
field classes you define in a model are actually stored in the Meta
class
when the model class is created (the precise details of how this is done are
unimportant here). This is because the field classes aren’t necessary when
you’re just creating and modifying attributes. Instead, they provide the
machinery for converting between the attribute value and what is stored in the
database or sent to the serializer.
Keep this in mind when creating your own custom fields. The Django Field
subclass you write provides the machinery for converting between your Python
instances and the database/serializer values in various ways (there are
differences between storing a value and using a value for lookups, for
example). If this sounds a bit tricky, don’t worry – it will become clearer in
the examples below. Just remember that you will often end up creating two
classes when you want a custom field:
Hand
class in our
example.Field
subclass. This is the class that knows
how to convert your first class back and forth between its permanent
storage form and the Python form.When planning your Field
subclass, first give some
thought to which existing Field
class your new field
is most similar to. Can you subclass an existing Django field and save yourself
some work? If not, you should subclass the Field
class, from which everything is descended.
Initializing your new field is a matter of separating out any arguments that are
specific to your case from the common arguments and passing the latter to the
__init__()
method of
Field
(or your parent class).
In our example, we’ll call our field HandField
. (It’s a good idea to call
your Field
subclass <Something>Field
, so it’s
easily identifiable as a Field
subclass.) It doesn’t
behave like any existing field, so we’ll subclass directly from
Field
:
from django.db import models
class HandField(models.Field):
description = "A hand of cards (bridge style)"
def __init__(self, *args, **kwargs):
kwargs['max_length'] = 104
super(HandField, self).__init__(*args, **kwargs)
Our HandField
accepts most of the standard field options (see the list
below), but we ensure it has a fixed length, since it only needs to hold 52
card values plus their suits; 104 characters in total.
Note
Many of Django’s model fields accept options that they don’t do anything
with. For example, you can pass both
editable
and
auto_now
to a
django.db.models.DateField
and it will simply ignore the
editable
parameter
(auto_now
being set implies
editable=False
). No error is raised in this case.
This behavior simplifies the field classes, because they don’t need to check for options that aren’t necessary. They just pass all the options to the parent class and then don’t use them later on. It’s up to you whether you want your fields to be more strict about the options they select, or to use the simpler, more permissive behavior of the current fields.
Field.
__init__
()¶The __init__()
method takes the following
parameters:
verbose_name
name
primary_key
max_length
unique
blank
null
db_index
rel
: Used for related fields (like
ForeignKey
). For advanced use only.default
editable
serialize
: If False
, the field will
not be serialized when the model is passed to Django’s serializers. Defaults to True
.unique_for_date
unique_for_month
unique_for_year
choices
help_text
db_column
db_tablespace
: Only for index creation, if the
backend supports tablespaces. You can usually
ignore this option.auto_created
: True if the field was
automatically created, as for the OneToOneField used by model
inheritance. For advanced use only.All of the options without an explanation in the above list have the same meaning they do for normal Django fields. See the field documentation for examples and details.
SubfieldBase
metaclass¶django.db.models.
SubfieldBase
¶As we indicated in the introduction, field subclasses are often needed for two reasons: either to take advantage of a custom database column type, or to handle complex Python types. Obviously, a combination of the two is also possible. If you’re only working with custom database column types and your model fields appear in Python as standard Python types direct from the database backend, you don’t need to worry about this section.
If you’re handling custom Python types, such as our Hand
class, we need to
make sure that when Django initializes an instance of our model and assigns a
database value to our custom field attribute, we convert that value into the
appropriate Python object. The details of how this happens internally are a
little complex, but the code you need to write in your Field
class is
simple: make sure your field subclass uses a special metaclass:
For example:
class HandField(models.Field):
description = "A hand of cards (bridge style)"
__metaclass__ = models.SubfieldBase
def __init__(self, *args, **kwargs):
# ...
This ensures that the to_python()
method, documented below, will always
be called when the attribute is initialized.
If you use SubfieldBase
, to_python()
will be called every time an instance of the field is assigned a
value. This means that whenever a value may be assigned to the field,
you need to ensure that it will be of the correct datatype, or that
you handle any exceptions.
This is especially important if you use ModelForms. When saving a ModelForm, Django will use form values to instantiate model instances. However, if the cleaned form data can’t be used as valid input to the field, the normal form validation process will break.
Therefore, you must ensure that the form field used to represent your
custom field performs whatever input validation and data cleaning is
necessary to convert user-provided form input into a
to_python()-compatible model field value. This may require writing a
custom form field, and/or implementing the formfield()
method on
your field to return a form field class whose to_python() returns the
correct datatype.
Field.
description
¶As always, you should document your field type, so users will know what it is.
In addition to providing a docstring for it, which is useful for developers,
you can also allow users of the admin app to see a short description of the
field type via the django.contrib.admindocs application. To do this simply provide
descriptive text in a description
class attribute of your custom field. In
the above example, the description displayed by the admindocs
application for a HandField
will be ‘A hand of cards (bridge style)’.
Once you’ve created your Field
subclass and set up
the __metaclass__
, you might consider overriding a few standard methods,
depending on your field’s behavior. The list of methods below is in
approximately decreasing order of importance, so start from the top.
Field.
db_type
(self, connection)¶connection
argument was added to support multiple databases.Returns the database column data type for the Field
,
taking into account the connection object, and the settings associated with it.
Say you’ve created a PostgreSQL custom type called mytype
. You can use this
field with Django by subclassing Field
and implementing the
db_type()
method, like so:
from django.db import models
class MytypeField(models.Field):
def db_type(self, connection):
return 'mytype'
Once you have MytypeField
, you can use it in any model, just like any other
Field
type:
class Person(models.Model):
name = models.CharField(max_length=80)
gender = models.CharField(max_length=1)
something_else = MytypeField()
If you aim to build a database-agnostic application, you should account for
differences in database column types. For example, the date/time column type
in PostgreSQL is called timestamp
, while the same column in MySQL is called
datetime
. The simplest way to handle this in a db_type()
method is to check the connection.settings_dict['ENGINE']
attribute.
For example:
class MyDateField(models.Field):
def db_type(self, connection):
if connection.settings_dict['ENGINE'] == 'django.db.backends.mysql':
return 'datetime'
else:
return 'timestamp'
The db_type()
method is only called by Django when the framework
constructs the CREATE TABLE
statements for your application – that is,
when you first create your tables. It’s not called at any other time, so it can
afford to execute slightly complex code, such as the
connection.settings_dict
check in the above example.
Some database column types accept parameters, such as CHAR(25)
, where the
parameter 25
represents the maximum column length. In cases like these,
it’s more flexible if the parameter is specified in the model rather than being
hard-coded in the db_type()
method. For example, it wouldn’t make much
sense to have a CharMaxlength25Field
, shown here:
# This is a silly example of hard-coded parameters.
class CharMaxlength25Field(models.Field):
def db_type(self, connection):
return 'char(25)'
# In the model:
class MyModel(models.Model):
# ...
my_field = CharMaxlength25Field()
The better way of doing this would be to make the parameter specifiable at run
time – i.e., when the class is instantiated. To do that, just implement
django.db.models.Field.__init__()
, like so:
# This is a much more flexible example.
class BetterCharField(models.Field):
def __init__(self, max_length, *args, **kwargs):
self.max_length = max_length
super(BetterCharField, self).__init__(*args, **kwargs)
def db_type(self, connection):
return 'char(%s)' % self.max_length
# In the model:
class MyModel(models.Model):
# ...
my_field = BetterCharField(25)
Finally, if your column requires truly complex SQL setup, return None
from
db_type()
. This will cause Django’s SQL creation code to skip
over this field. You are then responsible for creating the column in the right
table in some other way, of course, but this gives you a way to tell Django to
get out of the way.
Field.
to_python
(self, value)¶Converts a value as returned by your database (or a serializer) to a Python object.
The default implementation simply returns value
, for the common case in
which the database backend already returns data in the correct format (as a
Python string, for example).
If your custom Field
class deals with data structures
that are more complex than strings, dates, integers or floats, then you’ll need
to override this method. As a general rule, the method should deal gracefully
with any of the following arguments:
Hand
in our ongoing example).In our HandField
class, we’re storing the data as a VARCHAR field in the
database, so we need to be able to process strings and Hand
instances in
to_python()
:
import re
class HandField(models.Field):
# ...
def to_python(self, value):
if isinstance(value, Hand):
return value
# The string case.
p1 = re.compile('.{26}')
p2 = re.compile('..')
args = [p2.findall(x) for x in p1.findall(value)]
return Hand(*args)
Notice that we always return a Hand
instance from this method. That’s the
Python object type we want to store in the model’s attribute.
Remember: If your custom field needs the to_python()
method to be
called when it is created, you should be using The SubfieldBase metaclass
mentioned earlier. Otherwise to_python()
won’t be called
automatically.
Field.
get_prep_value
(self, value)¶get_db_prep_value()
This is the reverse of to_python()
when working with the
database backends (as opposed to serialization). The value
parameter is the current value of the model’s attribute (a field has
no reference to its containing model, so it cannot retrieve the value
itself), and the method should return data in a format that has been
prepared for use as a parameter in a query.
This conversion should not include any database-specific
conversions. If database-specific conversions are required, they
should be made in the call to get_db_prep_value()
.
For example:
class HandField(models.Field):
# ...
def get_prep_value(self, value):
return ''.join([''.join(l) for l in (value.north,
value.east, value.south, value.west)])
Field.
get_db_prep_value
(self, value, connection, prepared=False)¶connection
and prepared
arguments were added to support multiple databases.Some data types (for example, dates) need to be in a specific format
before they can be used by a database backend.
get_db_prep_value()
is the method where those conversions should
be made. The specific connection that will be used for the query is
passed as the connection
parameter. This allows you to use
backend-specific conversion logic if it is required.
The prepared
argument describes whether or not the value has
already been passed through get_prep_value()
conversions. When
prepared
is False, the default implementation of
get_db_prep_value()
will call get_prep_value()
to do
initial data conversions before performing any database-specific
processing.
Field.
get_db_prep_save
(self, value, connection)¶connection
argument was added to support multiple databases.Same as the above, but called when the Field value must be saved to
the database. As the default implementation just calls
get_db_prep_value()
, you shouldn’t need to implement this method
unless your custom field needs a special conversion when being saved
that is not the same as the conversion used for normal query
parameters (which is implemented by get_db_prep_value()
).
Field.
pre_save
(self, model_instance, add)¶This method is called just prior to get_db_prep_save()
and should return
the value of the appropriate attribute from model_instance
for this field.
The attribute name is in self.attname
(this is set up by
Field
). If the model is being saved to the database
for the first time, the add
parameter will be True
, otherwise it will be
False
.
You only need to override this method if you want to preprocess the value
somehow, just before saving. For example, Django’s
DateTimeField
uses this method to set the attribute
correctly in the case of auto_now
or
auto_now_add
.
If you do override this method, you must return the value of the attribute at the end. You should also update the model’s attribute if you make any changes to the value so that code holding references to the model will always see the correct value.
As with value conversions, preparing a value for database lookups is a two phase process.
Field.
get_prep_lookup
(self, lookup_type, value)¶get_db_prep_lookup()
get_prep_lookup()
performs the first phase of lookup preparation,
performing generic data validity checks
Prepares the value
for passing to the database when used in a lookup (a
WHERE
constraint in SQL). The lookup_type
will be one of the valid
Django filter lookups: exact
, iexact
, contains
, icontains
,
gt
, gte
, lt
, lte
, in
, startswith
, istartswith
,
endswith
, iendswith
, range
, year
, month
, day
,
isnull
, search
, regex
, and iregex
.
Your method must be prepared to handle all of these lookup_type
values and
should raise either a ValueError
if the value
is of the wrong sort (a
list when you were expecting an object, for example) or a TypeError
if
your field does not support that type of lookup. For many fields, you can get
by with handling the lookup types that need special handling for your field
and pass the rest to the get_db_prep_lookup()
method of the parent class.
If you needed to implement get_db_prep_save()
, you will usually need to
implement get_prep_lookup()
. If you don’t, get_prep_value
will be
called by the default implementation, to manage exact
, gt
, gte
,
lt
, lte
, in
and range
lookups.
You may also want to implement this method to limit the lookup types that could be used with your custom field type.
Note that, for range
and in
lookups, get_prep_lookup
will receive
a list of objects (presumably of the right type) and will need to convert them
to a list of things of the right type for passing to the database. Most of the
time, you can reuse get_prep_value()
, or at least factor out some common
pieces.
For example, the following code implements get_prep_lookup
to limit the
accepted lookup types to exact
and in
:
class HandField(models.Field):
# ...
def get_prep_lookup(self, lookup_type, value):
# We only handle 'exact' and 'in'. All others are errors.
if lookup_type == 'exact':
return self.get_prep_value(value)
elif lookup_type == 'in':
return [self.get_prep_value(v) for v in value]
else:
raise TypeError('Lookup type %r not supported.' % lookup_type)
Field.
get_db_prep_lookup
(self, lookup_type, value, connection, prepared=False)¶connection
and prepared
arguments were added to support multiple databases.Performs any database-specific data conversions required by a lookup.
As with get_db_prep_value()
, the specific connection that will
be used for the query is passed as the connection
parameter.
The prepared
argument describes whether the value has already been
prepared with get_prep_lookup()
.
Field.
formfield
(self, form_class=forms.CharField, **kwargs)¶Returns the default form field to use when this field is displayed in a model.
This method is called by the ModelForm
helper.
All of the kwargs
dictionary is passed directly to the form field’s
Field__init__()
method. Normally, all you need to do is
set up a good default for the form_class
argument and then delegate further
handling to the parent class. This might require you to write a custom form
field (and even a form widget). See the forms documentation for information about this, and take a look at the code in
django.contrib.localflavor
for some examples of custom widgets.
Continuing our ongoing example, we can write the formfield()
method as:
class HandField(models.Field):
# ...
def formfield(self, **kwargs):
# This is a fairly standard way to set up some defaults
# while letting the caller override them.
defaults = {'form_class': MyFormField}
defaults.update(kwargs)
return super(HandField, self).formfield(**defaults)
This assumes we’ve imported a MyFormField
field class (which has its own
default widget). This document doesn’t cover the details of writing custom form
fields.
Field.
get_internal_type
(self)¶Returns a string giving the name of the Field
subclass we are emulating at the database level. This is used to determine the
type of database column for simple cases.
If you have created a db_type()
method, you don’t need to worry about
get_internal_type()
– it won’t be used much. Sometimes, though, your
database storage is similar in type to some other field, so you can use that
other field’s logic to create the right column.
For example:
class HandField(models.Field):
# ...
def get_internal_type(self):
return 'CharField'
No matter which database backend we are using, this will mean that syncdb
and other SQL commands create the right column type for storing a string.
If get_internal_type()
returns a string that is not known to Django for
the database backend you are using – that is, it doesn’t appear in
django.db.backends.<db_name>.creation.DATA_TYPES
– the string will still be
used by the serializer, but the default db_type()
method will return
None
. See the documentation of db_type()
for reasons why this might be
useful. Putting a descriptive string in as the type of the field for the
serializer is a useful idea if you’re ever going to be using the serializer
output in some other place, outside of Django.
Field.
value_to_string
(self, obj)¶This method is used by the serializers to convert the field into a string for
output. Calling Field._get_val_from_obj(obj)()
is the best way to get the
value to serialize. For example, since our HandField
uses strings for its
data storage anyway, we can reuse some existing conversion code:
class HandField(models.Field):
# ...
def value_to_string(self, obj):
value = self._get_val_from_obj(obj)
return self.get_db_prep_value(value)
Writing a custom field can be a tricky process, particularly if you’re doing complex conversions between your Python types and your database and serialization formats. Here are a couple of tips to make things go more smoothly:
django/db/models/fields/__init__.py
) for inspiration. Try to find
a field that’s similar to what you want and extend it a little bit,
instead of creating an entirely new field from scratch.__str__()
or __unicode__()
method on the class you’re
wrapping up as a field. There are a lot of places where the default
behavior of the field code is to call
force_unicode()
on the value. (In our
examples in this document, value
would be a Hand
instance, not a
HandField
). So if your __unicode__()
method automatically
converts to the string form of your Python object, you can save yourself
a lot of work.FileField
subclass¶In addition to the above methods, fields that deal with files have a few other
special requirements which must be taken into account. The majority of the
mechanics provided by FileField
, such as controlling database storage and
retrieval, can remain unchanged, leaving subclasses to deal with the challenge
of supporting a particular type of file.
Django provides a File
class, which is used as a proxy to the file’s
contents and operations. This can be subclassed to customize how the file is
accessed, and what methods are available. It lives at
django.db.models.fields.files
, and its default behavior is explained in the
file documentation.
Once a subclass of File
is created, the new FileField
subclass must be
told to use it. To do so, simply assign the new File
subclass to the special
attr_class
attribute of the FileField
subclass.
In addition to the above details, there are a few guidelines which can greatly improve the efficiency and readability of the field’s code.
ImageField
(in
django/db/models/fields/files.py
) is a great example of how to
subclass FileField
to support a particular type of file, as it
incorporates all of the techniques described above.Sep 27, 2017