AssertionError: assert(len(cstruct) == len(field.sequence_fields))

[thomie]

Running the following program results in an AssertionError in deform/widget.py.

Uncommenting either of the two commented out lines prevents the problem.

import colander
from colander import SchemaNode, Integer
import deform
from deform.widget import Select2Widget


class TitleSchema(colander.MappingSchema):
    @colander.instantiate()
    class xs(colander.SequenceSchema):
        x = SchemaNode(Integer())

    y = SchemaNode(Integer())



def test_stackoverflow():
    schema = TitleSchema()
    form = deform.Form(schema)

    items = [
        ('__start__', 'xs:sequence'), ('x', '1'), ('__end__', 'xs:sequence'),
        # ('y', '2'),
    ]

    try:
        title = form.validate(items)
    except deform.ValidationFailure as e:
        title = {}

    # form = deform.Form(schema)
    form.render(title)

test_stackoverflow()

$ python ~/deform-stackoverflow.py
Traceback (most recent call last):
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/template.py", line 171, in render
    self._render(stream, econtext, rcontext)
  File "mapping_item_8ef2b9ece625905633682437b4262fd4.py", line 527, in render
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/field.py", line 512, in serialize
    return self.widget.serialize(**values)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/widget.py", line 1524, in serialize
    assert(len(cstruct) == len(field.sequence_fields))
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/thomas/deform-stackoverflow.py", line 34, in <module>
    test_stackoverflow()
  File "/home/thomas/deform-stackoverflow.py", line 31, in test_stackoverflow
    rendered_form = form.render(title)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/field.py", line 551, in render
    html = self.serialize(cstruct, **kw)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/field.py", line 512, in serialize
    return self.widget.serialize(**values)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/widget.py", line 1369, in serialize
    return field.renderer(template, **values)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/template.py", line 85, in __call__
    return self.load(template_name)(**kw)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/template.py", line 124, in __call__
    return self.render(**kwargs)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/zpt/template.py", line 297, in render
    return super(PageTemplate, self).render(**_kw)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/template.py", line 171, in render
    self._render(stream, econtext, rcontext)
  File "form_69db4de6f2356e2bc79900fcf934ee09.py", line 634, in render
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/field.py", line 741, in render_template
    return self.renderer(template, **values)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/template.py", line 85, in __call__
    return self.load(template_name)(**kw)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/template.py", line 124, in __call__
    return self.render(**kwargs)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/zpt/template.py", line 297, in render
    return super(PageTemplate, self).render(**_kw)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/template.py", line 191, in render
    raise_with_traceback(exc, tb)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/utils.py", line 75, in raise_with_traceback
    raise exc
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/chameleon/template.py", line 171, in render
    self._render(stream, econtext, rcontext)
  File "mapping_item_8ef2b9ece625905633682437b4262fd4.py", line 527, in render
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/field.py", line 512, in serialize
    return self.widget.serialize(**values)
  File "/home/thomas/.virtualenvs/vodsuite/lib/python3.6/site-packages/deform/widget.py", line 1524, in serialize
    assert(len(cstruct) == len(field.sequence_fields))
AssertionError: <unprintable AssertionError object>

 - Expression: "field.serialize(cstruct).strip()"
 - Filename:   ... python3.6/site-packages/deform/templates/mapping_item.pt
 - Location:   (line 29: col 34)
 - Source:     ... place="structure field.serialize(cstruct).strip()"
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Expression: "child.render_template(item_template)"
 - Filename:   ... ite/lib/python3.6/site-packages/deform/templates/form.pt
 - Location:   (line 46: col 32)
 - Source:     ... ace="structure child.render_template(item_template)"/>
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Arguments:  field: <Field - at 0x7f9579ebd668>
               cstruct: <_null - at 0x7f957be26b00>
               target_language: <NoneType - at 0x9beb80>
               repeat: {...} (0)
               error_class: <NoneType - at 0x9beb80>
               description:
               title: Xs
               oid: deformField1
               hidden: False
               category: default
               structural: False
               required: True
               input_prepend: <NoneType - at 0x9beb80>
               input_append: <NoneType - at 0x9beb80>

...

Versions used:

• python3.4 and python3.6
• deform 2.0.4
• colander 1.4
• Chameleon: 3.2

[kalbermattenm]

I just ran into the same problem (Python 3.6, deform 2.0.5, same versions for the rest).

It made my Python pserve instance crash completely.

I am wondering what is the cause of the issue...

[kalbermattenm]

Actually, this appeared because my model was defining a field which did not exist in the database, so my fault, sorry for the noise.

[thomie]

deform shouldn't crash though, but instead show a nice error message

[kalbermattenm]

Yeah... I though exactly the same, but from my point of view, it is not a blocker, it a missing coverage...

[stevepiercy]

@thomie @kalbermattenm I recently took on the role of maintainer for Deform. I'd accept a PR that returns a nice error message and provides a test. Here's one option to consider:

deform/deform/widget.py

Lines 1426 to 1428 in 3329bb1

	def serialize(self, field, cstruct, **kw):
	if cstruct in (null, None):
	cstruct = {}

and add right after that:

        if field.error is not None:
            raise field.error

=> colander.Invalid: {'y': 'Required'}

Would that do what you want?

[kalbermattenm]

Would that do what you want?

Yes, something like that... I lost a bit track of it, so I am not sure where exactly to put it...

[thomie]

Yes, something like that... I lost a bit track of it, so I am not sure where exactly to put it...

Same for me. I've not used deform for a while.

@stevepiercy: thanks for taking over maintainership.

[stevepiercy]

Cool! Well, if y'all want to try it out, place those two lines of code in your checkout of Deform in deform/deform/widget.py, immediately after line 1428.

Would y'all please submit a PR with a test to cover this, using your excellent example above?

We have overhauled how to get started with Deform, too, making it much easier to be a contributor. See https://github.com/Pylons/deform/blob/master/contributing.md for detailed step-by-step instructions. If you get stuck, I'm here to help.

Finally I'm doing tiny incremental releases of Deform now, so there's a good chance I will include contributions within a day or two of them getting merged to master and backported to 2.0-branch.