UTF-16 files not properly found

[donjoergo]

I noticed a weird behavior after upgrading from todo-tree to better-todo-tree.

The TODOs in SQL files are not shown in the extension's sidebar view anymore.

However it seems that when a SQL file is opened directly, all TODOs are highlighted correctly inside of the file. With Gruntfuggly/todo-tree this worked previously.

Can someone confirm, or maybe I am missing a configuration setting here?

The regex pattern:
(//|#|<!--|;|/\\*|^|--|^[ \\t]*(-|\\d+.))\\s*($TAGS)

The whole config, just for reference:

"better-todo-tree.tree.scanMode": "workspace",
  "better-todo-tree.general.showActivityBarBadge": true,
  "better-todo-tree.tree.showCountsInTree": true,
  "better-todo-tree.regex.regex": "(//|#|<!--|;|/\\*|^|--|^[ \\t]*(-|\\d+.))\\s*($TAGS)",
  "better-todo-tree.regex.regexCaseSensitive": false,
  "better-todo-tree.general.tags": [
    "BUG",
    "HACK",
    "FIXME",
    "TODO",
    "XXX",
    "[ ]",
    "[x]",
    "WORKAROUND"
  ],
  "better-todo-tree.highlights.customHighlight": {
    "BUG": {
      "icon": "bug",
      "gutterIcon": true,
      "foreground": "#ffffff",
      "background": "#ff1040",
      "iconColour": "#ff1040",
      "rulerColour": "#ff1040"
    },
    "FIXME": {
      "icon": "flame",
      "gutterIcon": true,
      "foreground": "#ffffff",
      "background": "#ff7810",
      "iconColour": "#ff7810",
      "rulerColour": "#ff7810"
    },
    "TODO": {
      "icon": "checklist",
      "gutterIcon": true,
      "foreground": "#ffffff",
      "background": "#ff1040",
      "iconColour": "#ff1040",
      "rulerColour": "#ff1040"
    },
    "XXX": {
      "gutterIcon": true,
      "foreground": "#ffffff",
      "background": "#8402e7",
      "iconColour": "#8402e7",
      "rulerColour": "#8402e7"
    },
    "WORKAROUND": {
      "icon": "tools",
      "gutterIcon": true,
      "foreground": "#2f2f2f",
      "background": "#02e7cc",
      "iconColour": "#02e7cc",
      "rulerColour": "#02e7cc"
    }
  }

[FALLAI-Denis]

Hi,

The default for Better To Do Tree is:

--> No SQL comment managment.

The proposed solution for SQL is:

---

This pattern works for us (COBOL and SQL):

    //"todo-tree.regex.regex": "(//|#|\u003c!--|;|/\\*|\\*\u003e|^......\\*|--)\\s*($TAGS)",
    "better-todo-tree.regex.regex": "(//|#|<!--|;|/\\*|\\*>|^......\\*|--)\\s*($TAGS)",
    "better-todo-tree.regex.regexCaseSensitive": false,

We do'nt have |^| in our regex.
It remains to be seen how the ^ character, with nothing following it, is interpreted in the regex.

[donjoergo]

Thanks for you comment @FALLAI-Denis.
Actually our regex config is the same as the recommended one, with one addition: --|

So that we can write comments like this:
-- TODO this needs to be fixed

I copied you config and tried it out. This one also didn't work.
I created a new project with one example file, and there suddenly it was working. And I thought: "How is that possible?"
So I dug deeper.

TLDR;
In the end I could narrow it down to the file encoding.
The error occurs with these encodings:

• ANSI
• UTF-16-LE BOM
• UTF-16-BE BOM

It's working fine with the following encodings:

• UTF-8
• UTF-8 BOM

So to me it seems that there is a real issue with better-todo-tree and not just a config issue.
I changed the bug's title and also added a screenshot of the error, to make it more clearer.