Decide on the target audience and project maturity level

[charlesfrye]

First-time hackers, early start-ups, or enterprise teams? An MVP or a five 9s webservice?

[charlesfrye]

I think we want to basically pitch this at the same level/audience as the FSDL content. This is for an agile, small team that's reasonably savvy with Python -- especially for ML but not necessarily for web dev.

So we want to minimize the number of services that need to be run -- independently managing and scaling five services for one project is not a good fit for this kind of team. We should also be cost-sensitive, since both hobby projects and small startups are .

It should, however, be as easy as possible to adapt the project to include more complex tooling. For example, we might even include a version of the template that uses a pipeline orchestrator to automate the entire process, even if that's not the baseline.

[vin00d]

I personally love nbdev, and use it extensively. I also acknowledge its perceived limitations (for which there are perfectly good workarounds).
For me its best features are:

• Ease of PyPi and conda packaging.
• Ease of up-to-date documentation & testing every step of the way.

So high-level question for this project / template is - how to retain many of the benefits of nbdev while also addressing its limitations.

In short - can we look at nbdev as a worthy competitor for this template / make this a compelling alternative to nbdev.

[ma7dev]

I really admire the structure of the open-source libraries we use on a daily basis. I think their structure could be utilized in many audiences as a good baseline to start their awesome projects.

I think we can set up a codebase template for researchers and open-source contributors for ease of adaption, which includes proper testing, packaging, documentation, etc. to standardize ML project development. In addition, include a tutorial-like for new-comers to understand the structure of the codebase in further detail.

[charlesfrye]

Yes! We just want to avoid going too far in the direction of writing things as though it was an open-source project.

For example, I think building a documentation website with sphinx/autodoc would be a bad idea. Many corporate projects would require authentication to view, which adds a lot of complexity, and docs require maintenance. Maybe something like nbdev that closes the gap between coding and documentation could make sense.

[FarisHijazi]

I think that the best way to have things done is by having a running example and then users would modify anything needed to their needs
This example should minimize hardcoding, for example maybe pass a dictionary of daya and labels so that it would be easy to support multiple inputs (this example is questionable but you know ehat i mean).

Something similar to pytorch lightning and then we just build an eco system around that.

I LOVE that fsdl chose an OCR as the project, because it has many components, we could make another OCR here in this project and i would personally be very interested in the data annotation section (self training, active lraening, model in the loop, etc).
Also extending fsdl-text-recognizer, we could add a detection model in the beginning too.
Document understanding has sooo many methods, so if we just make a cookiecutter for document understanding, it would already be a success and could be transfered to other problems (simply because document understanding is so broad, from OCR to end2end solutions like donut).

[charlesfrye]

I love the idea of extending to cover more of document understanding!

There's a nice article from Hugging Face on it here: https://huggingface.co/blog/document-ai

[charlesfrye]

If you want to spearhead some of the work on this, lmk! I won't be focusing on extending the Text Recognizer until early 2023.

In particular, the first thing we'd need to do is figure out how to make Lightning and Hugging Face play nicely together. FWICT, there's just too much implemented in Hugging Face for it to make sense to reinvent the wheel (modeling libraries, preprocessing eg tokenizers, model weights on the Hub), but I do really like Lightning for training.

[FarisHijazi]

Let's decide what things should look like first and reach a consensus

To go for huggingface or pytorch lightning or both and etc.
And what usecases, most people i know just use huggingface pretrained models, so I'm not sure how much we should integrate hugginface cuz it could be another project on its own.

Tell me what u think

[FarisHijazi]

I just need to give everyone a heads up now, that I won't be able to participate much until the end of January since I have many deadlines before then, I can still participate every now and then without commitment

[EdAbati]

I completely agree with @charlesfrye.

Audience: same as FSDL. Small team of people with good understanding of ML and Python. I think the documentation could also be a bit more beginner-friendly. Everyone should feel like they have all the info to quickly get familiar with any tool that the cookiecutter supports.

Project maturity level: Complete enough so that one can build an ML app end-to-end, simple enough so that one doesn't have to learn about/spin up many tools to make this work. Completely agree with being cost-sensitive. Ideally, all the tools we choose should have at least a sensible free trial.