I'm inserting objects in bulk to MongoDB using Mongoengine like this (simplified code):
class Customer(Document):
first_name = StringField(max_length=100)
last_name = StringField(max_length=100)
address = StringField(required=True)
instances = [Customer(**{'name': 'John', 'last_name': 'Doe'})]
Customer.objects.insert(instances)
I'm bulk inserting using list Document instances created from dicts. As you can see on the above example, the Customer instance lacks required address field. However, it is saved in the MongoDB anyway. Then in the MongoDB there is a Customer document without the required field address.
My question: What can I do so that the schema is validated when bulk inserting? Is a for loop and calling instance.save() for each Customer instance my only alternative?
In the MongoEngine documentation it is written:
By design, MongoEngine strictly validates the documents right before they are inserted in MongoDB and makes sure they are consistent with the fields defined in your models.
so I'm confused why using objects.insert() does not validate the document.
However, later in the docs it is written:
Validation runs when calling .validate() or .save()
so maybe by design objects.insert() does not validate documents and before bulk inserting there should be a loop going through all instances and calling validate()?
When you call Customer.objects what's returned is actually a QuerySet. The QuerySet.insert (docs here) method has no option for validation unfortunately.
Your idea to pre-validate and bulk insert seems like the best option:
instances = []
for instance in [{'name': 'John', 'last_name': 'Doe'}]:
customer = Customer(**instance)
customer.validate()
instances.append(customer)
Customer.objects.insert(instances)
This is much faster than calling save on each instance.