Is it possible to enforce a space?

[unhammer]

Almost an inverse of #11, it'd be nice if we could have some way of enforcing a space.

If input is "a, b" (three lexical units) and the rule outputs two units "c b", then separable will notice that there's no space between the first two units, and uses that empty string as the space between the first two output units so we get "cb" instead of "c b".

$ cat b.lsx
<?xml version="1.0" encoding="UTF-8"?>
<dictionary type="separable">
  <alphabet></alphabet>
  <sdefs>
    <sdef n="ex" c="Exasperative"/>
    <sdef n="ir" c="Irritative"/>
 </sdefs>

  <pardefs>
    <pardef n="meh">
      <e><i><w/><t/><j/></i></e>
    </pardef>
  </pardefs>

  <section id="main" type="standard">
    <e>
      <p><l>a<t/><j/></l> <r></r></p>
      <p><l>,<t/><j/></l> <r></r></p>
      <p><l>b<t/><j/></l> <r></r></p>
      <p><l></l>          <r>c<s n="ex"/><j/></r></p>
      <p><l></l>          <r>d<s n="ir"/><j/></r></p>
    </e>
  </section>

</dictionary>


$ lsx-comp lr b.lsx b.bin
main@standard 17 19

$ echo '^a<ir>$^,<cm>$ ^b<ex>$' | lsx-proc b.bin
^c<ex>$^d<ir>$

Expected:

^c<ex>$ ^d<ir>$

[mr-martian]

Proposal:
As discussed in #11, it would be nice if <j/> could be + as elsewhere, so add <d/> as a word boundary. For backwards-compatibility, we could put the new behavior of <j/> behind a command-line flag or just regex-replace all the existing files.

To solve both this issue and #11, I propose the following:

XML symbol	transducer symbol	behavior
<d space="auto"/> or <d/>	<$>	use the next entry (possibly empty) in the blank queue (the current behavior of <j/>)
<d space="no"/>	<$->	do not pop a blank off the queue
<d space="yes"/>	<$_>	pop a blank off the queue and replace it by a space if it's empty

@ftyers, @jonorthwash, @unhammer - does this seem reasonable?

If this sounds good to everybody, it shouldn't take more than an hour or so to set up.

[unhammer]

Would work for me :-)

I'd prefer regex-replace, seems like something one can fairly safely do.

[unhammer]

Should we take three weeks of silence as consent then? :) would be nice to have a solution for this seemingly trivial issue …

[unhammer]

… what if I just want a space without ^$? If I do

    <e>
      <p> <l>,<s n="cm"/><d/></l> <r></r></p>
      <i>og<s n="cnjcoo"/><s n="clb"/><d/></i>
    </e>

to delete a comma, I get

$ lsx-comp lr apertium-dan-nor.nor-dan.lsx nor-dan.autoseq.bin && echo '^*foo$^,<cm>$ ^og<cnjcoo><clb>$' | lsx-proc nor-dan.autoseq.bin
main@standard 10 9
^*foo$^og<cnjcoo><clb>$

and the exact same whether I specify space=yes/no or not at all.

I can put <d space="yes"/> in the <r>, but then it of course outputs an empty ^$ (rtx-proc seems to drop it, but I'm guessing it could get in the way of the parse)

[mr-martian]

So the issue here is the handling of blanks at the boundaries of matches.

The hacky solution is just to make your lsx patterns 1 token longer.

The less-hacky solution is to come up with a way of representing the desired status of blanks in the FST.

And perhaps as a third level, we could make apertium-posttransfer more capable and have it redo spacing more generally.

[unhammer]

On a related note: Is there a good solution for enforcing no-final-space on a <par usage without duplicating the whole pardef tree? E.g. I have a rule that inserts a comma (should have no space in front of it) after a noun phrase. I have a noun phrase pardef that matches (adj)*(ncmp)*n|prn|(ncmp)*np|det.something via a multitude of other pardefs which all end in <d/> so (I don't have to add a final <d/> after each usage in a rule). Maybe I should just never end pardefs in the final <d/>, and explicitly add d's in the rule? Or is there a better way?

[mr-martian]

I can't think of a better way, though one thing you could do is

<pardef n="NP_space">
  <e><par n="NP"/><i><d/></i></e>
</pardef>
<pardef n="NP_nospace">
  <e><par n="NP"/><i><d space="no"/></i></e>
</pardef>

That would still require you to remove <d/> from a lot of pardefs, but without making you add it to all the rules.

For initial space, it occurs to me that I could special-case <d/> at runtime so that if it appears as the first symbol of the output it will manipulate spaces without creating an empty LU ... but then lsx-proc treats blanks as being attached to the preceding word. Drat.