split by paragrphs

[UrikFo]

I tried your application. It excellent split a solid block into words. But the problem is that sometimes I download pdf files which I want to convert to MS Word. Some of pdf files has been create without spaces. The majority of pdf convertors are not able to solve this problem (except Adobe Acriobat which distorted fonts) and online2pdf (keep fonts properly but has limits in file sizes).

Therefore I created a WinForm application on VB.NET where I added your library through Nuget with 2 multi-lined textBoxes (input & output). Text splitted excellent but I got a text not splitted by paragraphs.

The second suggestion (I am not sure if it is possibly). Some portion of text is bold-faced or underlined. Threfore, I have to copy these portions of text separately or after I will paste a result I will change the font manually. I am not sure if it is possible to keep the portion of text in RTF format.

I apologize for taking your time.

[UrikFo]

By the way some words still not recognized because author did not use proper English words (something like "Yessss!!!!). Therefore, your recognition is not 100% corect bu approx 90-95%. I am not sure if you able to improve your algorithm....

SymsSpellSample.txt

[wolfgarbe]

I got a text not splitted by paragraphs

Do you mean that SymSpell doesn't respect U+2029 (Paragraph Separator)? Then you can do the splitting in a preprocessing step, and word segment the parts separately.

Some portion of text is bold-faced or underlined

SymSpell currently doesn't preserve any format information. This would be useful, but is beyond the scope of the project (which is merely intended as a showcase for the Symmetric Delete spelling correction algorithm, but not a turn-key solution)

Some words still not recognized because author did not use proper English words (something like "Yessss!!!!).

SymSpell is a probabilistic algorithm, based on a dictionary and its word frequencies (probabilities). As such, it will never be 100% accurate.

Sometimes one can segment the same string into different words. SymSpell will then chose the variant with the highest probability, but will never know for sure how the original segmentation really was.

As for your example "Yessss!!!!": You can add "yessss" to your dictionary. You could also detect and remove excessive replications of a character in a pre-processing step.

[UrikFo]

Sorry, I apologize for misunderstanding about U+2029. Is it my work to separate paragraphs? Or you mean just put 1 paragraph running only or if it is implemented please let me know. I understand that paragraph separater is 1 or 2 CRLF or \r\n (depends on your languaage you prefer)

I understand that it is a probabalistic algorithm. Therefore....