Things I do every time I start a Django project

(brntn.me)

222 points | by 8b16380d 12 days ago

26 comments

  • silviogutierrez 12 days ago
    If you want a batteries-included, zero-config Django + React framework, check out https://www.reactivated.io .

    It incorporates most of these best practices, along with React server side rendering in addition to regular Django templates.

    Full disclosure: I'm the creator.

    • xcambar 11 days ago
      Tangent comment on JSX/TS/JS and a snippet found on reactivated.io. Not a comment on the article or reactivated.io in any way.

      > export const Flavor = () => <Select options={flavors} />;

      When I read the above, I fear for the new devs. The number of concepts to grasps and the level of parsing and mind bending to do is incredible.

      I used to teach code, before arrow functions and React. And I am convinced I would have lost some students to this snippet, especially the "= () =>" segment.

      Are we gone too far syntactically for the sake of compactness rather than understandability, allin the name of (fake, IMO) readability?

      • WorldMaker 11 days ago
        Some of that shifts with syntax highlighting: drawing the eyes to which bits are more specifically related. The arrow => should highlight as a single operator (just as >= does), for instance. Most people call these "arrow functions" so thinking of "=>" as the arrow operator is a habit that quickly builds.

        They are controversial and everyone has different opinions, but I also think this is where programming ligatures come in extremely handy. When => looks more like ⇒ and is even more obviously an arrow, I think that also starts to make it easier to visually "parse" the flow of code like that.

        For what it is worth, arrow functions weren't added solely for compactness, but also to fix some historic issues with classic function syntax. (Lexically scoped `this` versus caller scoped `this` being the big one.) A new type of function syntax was desired anyway for those reasons, and the compact syntax was the icing on the cake.

        I haven't had the pleasure of teaching such things to students at this point in my career, but I have given a hand to many a junior developer to grasping some of these ideas and I don't think it is that tough, though it can be a shock/surprise if the last JS you touched was many years before. Especially if you are trying to also learn what JSX and/or TSX add on top of all the changes in ES2015+.

        • fuzzy2 11 days ago
          No. That's perfectly normal syntax you would use every day. It is not sophisticated or special.

          Also, consider functional languages like Haskell or whatever. Their syntax is exceptionally foreign for users of C-like languages, and so are the concepts. Still, they're (relatively) widely used.

          • xcambar 11 days ago
            Comparing something to something else harder does not move the former into the realm of easy things.

            Haskell though requires significantly more work than JS to grasp, for sure.

          • muspimerol 11 days ago
            If you work with JSX daily, which 99% of React developers do, this is not hard to grok. It's a very typical functional composition pattern in React. Since this framework is targeted at those who want to use React, and presumably already use it, I don't see the issue. It's a snippet to show how Reactivated lets you use the React ecosystem, not how to use React - that's what the docs are for.
            • easytiger 11 days ago
              > Are we gone too far syntactically for the sake of compactness rather than understandability, allin the name of (fake, IMO) readability?

              One of the reasons perl faded away, imho

              • true_religion 11 days ago
                I’m not sure that brand new devs who have trouble reading standard JSX, are exactly the target market for a bleeding edge project that combines two popular frameworks together.

                I think the target market is experienced people who want to use both react and Django, and feel that this project saves time on conceiving and writing their own integration.

                • sumy23 11 days ago
                  `() =>` is just the lambda syntax. JavaScript has, by far, one of the most understandable lambda syntaxes.

                  For instance, look how blocks are declared in Objective-C: http://fuckingblocksyntax.com/

                  • wruza 11 days ago
                    Compactness and understandability are the same thing on a large scale or under time pressure. This example is definitely loaded, but compared to more wordy alternatives it just cuts to the chase.
                    • xcambar 11 days ago
                      > Compactness and understandability are the same thing on a large scale

                      Could you elaborate on that, please? The way I read this, I could not be further from agreement :)

                      • wruza 11 days ago
                        Extended code reads like a narrative journalism. “It was an end to a long dark night and trees shyly stayed in a fog when options in a full entirety of their generous content got passed into a pure functional instance of a select tag”.
                        • xcambar 11 days ago
                          I appreciate your ~writing~ coding style ;)

                          Maybe we can find consensus by avoiding the extremes.

                          "Extended code" as you name it is certainly bad with regards to explicitness, while extreme compactness like (some) one liners are as bad as the former but with regards to readability.

                          Both are negatively impacting understandability.

                  • lordofgibbons 12 days ago
                    If you're doing server-side rendering, why do it in React SSR instead of Django templates?

                    I've seen this pattern a lot recently but haven't figured out what the extra complexity gets you

                    • __float 12 days ago
                      I haven't used this particular framework, but having type safety in React through writing TypeScript, generating bindings to the backend so those are typed as well.

                      There's also a _ton_ of preexisting React components you can generally drop in and use, which is less true these days with something like a Django template.

                      You also have the option of doing SSR and then doing more dynamic stuff on the client side, as a sort of optimization (and plain better user experience than making _more_ server requests to get the initial page state loaded).

                      • lordofgibbons 12 days ago
                        Thanks for taking the time to reply, but I'm still not really understanding the benefit.

                        Type safety is incredibly important if you're lots of logic in a language, which is why TS is great for SPAs. Does the type safety of TS get you anything if you're just doing SSR with not a whole lot of logic in TS? All of your application logic will be on the Python side of things.

                        >You also have the option of doing SSR and then doing more dynamic stuff on the client side

                        Isn't this just the old-school way of doing things before SPAs came around? i.e you render the page on the server and then add dynamics features using JS. I think the new way of doing this is with htmx, hotwire, etc.

                        • moomoo11 11 days ago
                          I think the biggest benefit of SSR is you get the first page fully rendered out (good for SEO), and beyond that page the react SPA takes over by doing all the cool client side stuff like routing and what-not.

                          The biggest benefit for SSR in my opinion is SEO + first load is fast because its already generated. After that though, its just the plain old SPA experience.

                      • sgt 11 days ago
                        It's because React (and other SPA technologies that also happen to work with SSR) is all the buzz. It doesn't actually necessarily make sense. The risk to a project is usually NOT the technology chosen for frontend.

                        Django templates are perfectly fine as long as you leverage template tags the way they were intended.

                        • silviogutierrez 11 days ago
                          I would agree that "Django templates are perfectly fine"... until: https://www.reactivated.io/documentation/philosophy-goals/#u...

                          Of course there's many ways around this, but doing them in a declarative, type-safe way is not trivial when using Django templates.

                          • lordofgibbons 11 days ago
                            Given that example, would you say using SSR with React to add bits of dynamic-ness is directly comparable with using templates + hotwire/htmx?

                            Or is it an apples to oranges comparison?

                            • silviogutierrez 11 days ago
                              It's very similar, except hotwire/htmx won't give you server rendering. It's not that big a deal if it's just small snippets.

                              This does make testing a lot easier, as the server response is identical to the hydrated client.

                              • ColonelPhantom 9 days ago
                                A hypertext approach such as HTMX in fact does give you server rendering, since you send to the client a normal SSR page with some extra annotations that add SPA features.

                                For React, SSR is secondary. For HTMX, it is primary.

                            • sgt 11 days ago
                              There's a lot of good advice in that article. I liked the part on cookies in particular. You rarely read that anywhere.
                          • rattray 11 days ago
                            You can use the same templates on the backend and frontend. That may not matter to you, at least not now, but it's a great reason in many contexts.

                            Personally I find JSX to be a terrific templating system, much better than anything like Django's, but if you're familiar with both and prefer Django's, this benefit does not apply to you!

                          • ttymck 11 days ago
                            Oh wow, this looks really good. I want to say, what I was really hoping to see is a "how does this work?" section. I'll read the source code, but it would be nice to have a quick narrative explanation.

                            EDIT: Looks like the "Concepts" page has what I am looking for. I would add some of that to the front page.

                            • greenie_beans 12 days ago
                              this is cool, thanks for sharing. this (https://www.reactivated.io/documentation/philosophy-goals/) fits my bias so i'm def gonna try it. curious how it works at a low level, too
                            • wanderingmind 11 days ago
                              Can you link any sample projects that currently uses reactivated?
                              • silviogutierrez 11 days ago
                                Sure:

                                The docs site itself: www.reactivated.io . Notice no JS loaded on the client side. It's purely SSR. The code is in the repo under /website.

                                My personal blog: www.silv.io

                                And my business: www.joyapp.com

                                I don't track others that use it, but hopefully they're out there!

                            • ryanisnan 12 days ago
                              This looks great!
                            • theptip 12 days ago
                              If you do this a lot you should look into ‘startapp —-template’ which lets you bake in boilerplate like this.

                              https://docs.djangoproject.com/en/4.0/ref/django-admin/#cmdo...

                              One minor point on app names. They are really hard/annoying to change once you have production data, because your DB tables will always be appname_modelname, and renaming tables is a real PITA. You can set the model to point to a nonstandard table, but then your project has a confusing non-Djangonic wart that will annoy you forever more. For this reason I think ‘core’ is a safer option, unless you are certain what your app should be named. Naming after one domain model is likely to be too specific. (I strongly recommend against multiple apps until you have functionality you need to commonize between projects, as refactoring models between apps is a nightmare if you get the boundaries wrong.)

                              • saila 12 days ago
                                Regarding apps, I was just thinking you could create a directory structure like this, essentially eschewing the whole concept of Django "apps":

                                    <project>
                                        src/
                                            <package>/
                                                models/
                                                    __init__.py    # Import all model classes (necessary for Django to find them)
                                                    some_model.py  # Define SomeModel class
                                                views/
                                                    __init__.py
                                                    some_view.py   # Views for SomeModel
                                                settings.py        # Add "<package>" to INSTALLED_APPS
                                                urls.py
                                
                                You still need to register the top level package as an app for Django to find things, but then you don't have to deal with Django's concept of apps beyond that. All your tables will be named <package>_<model> by default, which seems nice.

                                If it turns out later you need resusable "apps" for some reason, you could always add an apps/ subpackage for them.

                                I haven't tried this in a real project, so I don't know if there are any downsides, but it seems like a decent approach.

                                • theptip 12 days ago
                                  Yeah I've played around with this approach in the past while prototyping service boilerplates, I think it's viable. I never got it polished enough to publish as a startproject --template and ultimately went with Flask for microservices, so I didn't finish the prototype.

                                  I found this article useful while I was experimenting: https://simpleisbetterthancomplex.com/article/2017/08/07/a-m...

                                  There are a few different places where little things break and then you need to use an obscure settings.py variable to fix them, which makes me a little nervous since it will cause a little cognitive friction for Django-fluent developers joining your project. But I do think it's worth experimenting with.

                                • chomp 12 days ago
                                  > For this reason I think ‘core’ is a safer option, unless you are certain what your app should be named.

                                  Yep, this is what I do. It’s rare that I make anything that can be shared cross-app (most Django apps I make are super simple CRUD-types), so every one of my Django apps has a ‘core’. I also recommend this approach.

                                • nojs 12 days ago
                                  I would add to this list: convert your template engine to Jinja2. I held out for a long time on this, and now I would not go back. The purist approach ("keep logic out of templates") sounds good, but in practice leads to spaghetti hacks with weird additional template tags to do simple things, a bunch of additional context variables with no meaning, etc.
                                  • raverbashing 11 days ago
                                    > purist approach ("keep logic out of templates") sounds good, but in practice leads to spaghetti hacks

                                    This. Exactly this

                                    Go for Jinja2 and don't look back

                                    If there's one defect in Django is the rosy-colored purist view that permeates a lot of its design decisions.

                                    Keeping "logic only in code" made sense in the 90s. And no, I'm not building a new template tag every time I make some stylistic choice.

                                    • spapas82 11 days ago
                                      About the purist view on Django: The django templates don't annoy me that much, but what is really annoying is the avoidance of using thread locals in Django. This makes very difficult stuff that would be trivial in other frameworks (ie flask), like getting the current user.
                                    • 41b696ef1113 12 days ago
                                      As another hold-out on Jinja, what pushed you over the edge? Half of the appeal to me in Django is that if you follow the easy path, one project should look like another. Breaking conventions for a few template niceties has not seemed worth it (although, I am not a front end guy, and survive on very barebones presentation).
                                      • nojs 12 days ago
                                        > what pushed you over the edge?

                                        Using Tailwind without a frontend js framework. I needed a way to create lightweight “components” ala React but on the backend, because otherwise you’re either copy pasting classes like mad or (ab)using tailwind’s @apply.

                                        • spapas82 11 days ago
                                          I had used this project for creating components https://github.com/EmilStenstrom/django-components/

                                          Also, for simpler cases I include partial templates.

                                          • jereze 11 days ago
                                            How do you create your components in Jinja2? Using the include tag?
                                            • nojs 11 days ago
                                              I am using {% macro %}
                                              • drcongo 11 days ago
                                                I've been gradually removing all use of Jinja macros from our projects as they make debugging so much more painful. Everything now goes in template globals, which lets you debug them the same as anything else, gives far clearer stack traces and lets you do clever things with caching.
                                      • davepeck 12 days ago
                                        Straightforward, useful advice. I pretty much do all of these when I start new Django projects.

                                        The need for a custom User model makes me a little sad every time because, well, single `name` fields (rather than `first_name`, `last_name`) and email-for-username do feel to me like more sensible defaults circa 2022.

                                        • ethanpil 12 days ago
                                          I'm curious as to why you think a single name field is better than the traditional separate first and last?
                                          • Daishiman 12 days ago
                                            Per https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... , names have far too much ambiguity to consider that two fields is necessary or sufficient to encapsulate that information.
                                            • mdisc 11 days ago
                                              Maybe for consumer apps, but for a lot of enterprise apps that have to be compatible with existing APIs or domain-specific data standard that require two name fields, that’s not really a viable approach.
                                        • hsbauauvhabzb 12 days ago
                                          I’ve never really understood the concept of an ‘app’, in some projects I’m sure in makes sense but it not being transparent by default in a new project seems silly.

                                          Am I missing something here?

                                          • nojs 12 days ago
                                            I used to build everything with one giant “monoapp” and completely ignore that aspect of Django, which worked pretty well. Eventually though you need to split up your models into multiple files, and then your views, and admin, and so on, and it turns out there’s a lot of benefits of just segregating things into “apps” from the beginning. The only real downside I’ve noticed in practice is the omni-file-search feature in my IDE is harder to use, because you now have 7 “models.py” and it’s a bit harder to quickly navigate to the right one. If you split up files without using apps (just normal Python imports), you don't have the same problem with all the names being duplicated.

                                            The pitch that you can "reuse" apps across projects always seemed weird to me, because that's basically never something you actually want to do/can do easily without major surgery.

                                            • Doxin 11 days ago
                                              Protip on separating your models etc out: use modules instead of apps. so instead of a models.py have a models folder with a structure like this

                                                  - models
                                                      cheese.py
                                                      spam.py
                                                      __init__.py
                                              
                                              You can then put all your cheese related models in cheese.py, and all your spam related models in spam.py. Then Simply import those models from __init__.py and django is none the wiser that anything changes. Tada: organized models without having to dive into the broken mess that is apps.

                                              The same trick of course applies to any python file you want to split up. Views, urls, managers, etc etc.

                                              • nojs 11 days ago
                                                Yeah that works pretty well too. I think there’s some minor advantages of having them in apps, they’re sorted neatly in the admin, you’re forced to split views, models, templates, admin, etc together which keeps things a bit cleaner. But it’s also messier in other ways (you have to sift through too many files to find the thing you want).
                                                • Doxin 8 days ago
                                                  Migrations that cross app boundaries are a nightmare. Before you know it there's thousands of migrations in your various apps, and squash is powerless to help you. It'll pretend to work but break your migrations if you try to squash across any migration that depends on a migration in a different app.

                                                  IMO migrations in combinations with apps are fully broken in django. it's unworkable for large projects.

                                              • danpalmer 11 days ago
                                                Shameless plug, I've given a talk about scaling Django codebases to many apps at PyCon (2021) and PyCon UK (2017). At my previous job we had around 500 apps in our codebase and it was honestly great to work with.
                                              • 41b696ef1113 12 days ago
                                                I will sign onto this approach as well. Never expanded beyond a `core` app, which may have duplicated modules as the project grows (models_foo, models_bar, views_foo, views_bar, etc). I have never felt that any one component of the project was isolated enough from the rest of everything that it made sense to firewall off pieces into apps.
                                                • nojs 11 days ago
                                                  I’ve never successfully isolated them in the sense of having no cross imports, but I think there are advantages keeping the code broken up anyway.
                                              • knicholes 12 days ago
                                                Say you want to easily create a comments app or ratings that you can associate with any other model in your project. Write that app once and use in many places. Or maybe something that sucks to write that may be error-prone, like auth, especially social auth which may change frequently. Maintaining one app and sharing it is way better than everyone trying to keep up with the changes themselves.

                                                Or take a look at https://djangopackages.org/ for examples of apps people like to share.

                                                • edmundsauto 12 days ago
                                                  Can't you just keep the logic for your comments app in the monoapp and use it with everything?

                                                  I understand if you want to distribute your app for others to incorporate, but otherwise it felt like the documentation overemphasizes the "app" aspect. It's a reasonable structure but more of a guideline... unless you want to reuse in other codebases

                                                • zach_garwood 12 days ago
                                                  I think many (most?) Django apps only ever need one app. I think the fact that it's front and center in Django is a bit if a distraction that makes people think they _have_ to use multiple apps.
                                                  • switch007 12 days ago
                                                    100% agree. It’s a distraction
                                                  • BiteCode_dev 11 days ago
                                                    The name suck, but the idea is they are plugguable: check the django packages site for an idea of the benefit of that. It's essentially a plugin system with autoloading of some resources, such as the model and templates.

                                                    Also, apps override each other in the order or import. E.g one app can override the template of another just by using the same name. This means you can plug AND extend.

                                                  • focom 12 days ago
                                                    And also install Django Extensions (https://django-extensions.readthedocs.io/en/latest/)

                                                    It makes the Django CLI comparable to rails. Notably the reset_db and shell_plus command

                                                    • drcongo 11 days ago
                                                      shell_plus is fantastic.
                                                    • Apreche 12 days ago
                                                      Reasonable. Personally I just made a template project to get everything off the ground in one step, even the production environment and repository.

                                                      https://github.com/Apreche/djangogoboot

                                                      • instb3at 12 days ago
                                                        I don't know why no one mentioned this.

                                                        https://cookiecutter-django.readthedocs.io

                                                        I use a modified version of this for all my projects. This one comes with all the goodies

                                                        • pratyushmittal 12 days ago
                                                          Yup, a custom cookiecutter template saves ton of time.

                                                          I created this one to create a production ready Django project in few minutes: https://github.com/Mittal-Analytics/django-cookiecutter

                                                            Things this does:
                                                            - README.md: setup readme with development setup
                                                            - Django split settings: split settings for local, testing and production
                                                            - Split requirements: split requirements.txt for local and production
                                                            - Pre-commit hooks: setup pre-commit hooks for black and pyflakes
                                                            - django-envoiron: database config and secrets in environment
                                                            - editorconfig: sensible tab/space defaults for html, js and python files
                                                            - remote-setup: setup hosting on uberspace
                                                            - git push deployment: `git push live` makes the changes live
                                                            - github actions for tests: run tests automatically on Github
                                                        • ttymck 12 days ago
                                                          Really sensible, straightforward recommendations. Nothing flashy.

                                                          Honestly I think these should be defaults (or at least configurable options) of the project generator. I guess you could use cookiecutter, but it's not really worth it.

                                                          • kubami 11 days ago
                                                            This depends. If you start a new project often it is much better to use a template.
                                                          • awinter-py 12 days ago
                                                            7. completely rewrite the auth system because it's not sufficiently flexible, chafe at the boilerplate, then port the backend to a different framework + miss the admin system a little
                                                            • BiteCode_dev 11 days ago
                                                              Django-allauth usually got you covered, but in the rare case it doesn't, creating a micro service just for auth is a way better alternative than a complete rewrite.
                                                              • the_cat_kittles 11 days ago
                                                                painfully true
                                                              • BiteCode_dev 11 days ago
                                                                I do pretty much the same, which makes me think it should be django default startup template.

                                                                I would also do:

                                                                - install django_extensions and django_debug_toolbar

                                                                - rename the admin URL route to contain a uuid

                                                                - setup black, mypy, pylint, ipdb, ipython, dotenv and doit (I do so for all my python projects)

                                                                - manage.py createsuperuser

                                                                Also, very often :

                                                                - setup celery (locally with fs backend)

                                                                - setup vitejs

                                                                - install sentry and mailtrap plugins

                                                                - install django-allauth

                                                                - install DRF

                                                                And currently exploring:

                                                                - replace DRF with django ninja

                                                                • senko 11 days ago
                                                                  Using a boilerplate template can really speed things up, and more importantly, avoid forgetting to set up (or change from default) something that's hard to change later, like the user model.

                                                                  I do a lot of Django+DRF projects and built https://apibakery.com/ to help with the initial scaffolding and defining the models (started as my internal tool, later opened it up to the public).

                                                                  • awongh 11 days ago
                                                                    One thing I never figured out was dependency management. Some others are linking to two scoops of django and other starter templates, it seems like none of them get past a kludgy set or requirements.txt files. I want something easy to use like ruby bundler or npm. I want to be able to specify my dependencies and install them in one command, and specify if something is production or local in that same command.
                                                                    • h3ctic 11 days ago
                                                                      Have you tried Poetry? (https://python-poetry.org)
                                                                      • mrweasel 11 days ago
                                                                        Out of curiosity, and because I rather prefer pip and the requirements.txt file over npm or anything Ruby has to offer, what issues do you encounter with dependency management in Django/Python.

                                                                        Granted, trying to build your own reusable modules for Django is not a great experience. In these cases the tools feels rather restrictive.

                                                                        • Gordonjcp 11 days ago
                                                                          > I want to be able to specify my dependencies and install them in one command

                                                                          "pip install -r requirements.txt"

                                                                          If you have different requirements, you can use a different file.

                                                                          Obviously this is only for development, and if you package your Python code properly then you can install the whole damn thing as a package.

                                                                          • lmns 11 days ago
                                                                            I use pip-tools with venvs for that and can‘t complain. It pins dependencies and you can create different requirements.txts for local dev and production (production for me is usually dev minus some debugging tools). It‘s not particularly fancy, but it gets the job done.
                                                                            • Arcanum-XIII 11 days ago
                                                                              And a way to manage the update of them… currently doing this blindly, running the test suite after each updated package hoping nothings broken.

                                                                              Slow and frustration prone with “sorry this package require packagex version 1 but packagew require it to be 2 now”

                                                                            • adiyatmubarak 12 days ago
                                                                              I used Django, and personally I don't like Django startaproject command that not allow us to create project that contains dash "-". Also default appname has the same name with the project name can confusing the new comer in Django.

                                                                              I always structures my project like this when start my Django project.

                                                                              $ mkdir -p my-example-project/src

                                                                              $ cd my-example-project/src

                                                                              $ django-admin startproject youtube_downloader .

                                                                              so all Django related will be under src folder.

                                                                              • abalaji 12 days ago
                                                                                That's actually a Python convention issue rather than a Django issue. Python convention dictates preference for underscores. This is because dashes cannot be used in a module name.
                                                                                • adiyatmubarak 12 days ago
                                                                                  Yes, because default Django app name by default using the same name with Django project name.
                                                                              • rajasimon 12 days ago
                                                                                Every time whenever I start a new project.

                                                                                1. django-environ 2. celery 3. channels 4. apps structure https://rajasimon.io/blog/django-project-structure/

                                                                                To setup all the "Hello World" take long time.

                                                                                • xwowsersx 12 days ago
                                                                                  How often does one start a new Django project? I've been working on the same project for a couple years...
                                                                                  • BiteCode_dev 11 days ago
                                                                                    5 times a year if you are a freelancer.
                                                                                    • mod 12 days ago
                                                                                      I started dozens of rails projects when doing client work. We occasionally chose Django, too.
                                                                                    • kolanos 12 days ago
                                                                                      The first two can be covered by this helpful library. [0]

                                                                                      [0]: https://github.com/joke2k/django-environ

                                                                                      • guico 11 days ago
                                                                                        Why don't you make a project template instead and start from there?

                                                                                        That's what I do, complete will all the dependencies I always need (like Celery, Allauth, etc)

                                                                                        • revskill 11 days ago
                                                                                          Why not ... expressJS + typescript + React then ? Not sure what's the point of using Django, except for You only know Python reason
                                                                                          • nasir 11 days ago
                                                                                            Really irrelevant comment. Moreover, the "You only know Python" reason is a very strong argument.
                                                                                            • sajrar 11 days ago
                                                                                              It is almost like one could leverage python and all the amazing libraries to compute interesting things.

                                                                                              It is almost like we can actually build web apps that do more than CRUD.

                                                                                            • andrewingram 11 days ago
                                                                                              Short answer: Django comes with so much out of the box that you end up reinventing a lot of wheels if you go with a pure JavaScript/TypeScript approach.
                                                                                            • zmdgg 12 days ago
                                                                                              all great bits of advice except the first.

                                                                                              keep your secrets out of your environment! especially when you aren't providing any additional safety advice like making sure DEBUG is off

                                                                                              • digisign 12 days ago
                                                                                                Some reasons:

                                                                                                - https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-... - https://blog.forcesunseen.com/stop-storing-secrets-in-enviro...

                                                                                                They both recommend mounting them into a tmpfs, and then unmounting, but I've not yet tried this.

                                                                                                • josephcsible 11 days ago
                                                                                                  Environment variables exist only in memory and can't be read by any user other than the process's own and root. There's nothing wrong with keeping secrets there.
                                                                                                  • BiteCode_dev 11 days ago
                                                                                                    Depends of the size of your setup. Most website are small, and their threat model don't require more than having the secrets in a systemd file env statement.

                                                                                                    After all, if you have a single server, and the attacker can read a root protected file oe the spawned processes context, you are pwned already. As for exposing env var, popping os.environ is usually enough.

                                                                                                    No need to pay for more than you must: bots and script kiddies are not mossad.

                                                                                                    • robervin 12 days ago
                                                                                                      I've heard this advice a number of times, but often run up against otherwise standard looking systems that rely on secrets in environment variables -- mainly thinking about AWS's requirement when using Secrets Manager with ECS; the secrets are stored securely, but ultimately loaded into a containers environment.
                                                                                                      • runnerup 12 days ago
                                                                                                        Where should you put secrets instead? Obviously not the filesystem.
                                                                                                        • thaumasiotes 12 days ago
                                                                                                          > Obviously not the filesystem.

                                                                                                          How would you avoid that?

                                                                                                          Even this article, which recommends keeping your secrets in environment variables, tells you to implement that by storing them in the filesystem. The advice isn't to avoid storing your secrets in the filesystem; it's to avoid storing them in version control.

                                                                                                          It is apparently an exercise for the reader to figure out why it's better to read your secrets out of the environment, which read them from a file you provided, than to read them from your own file yourself.

                                                                                                          • BiteCode_dev 11 days ago
                                                                                                            Linux already stores plenty of secrets in files in /etc. Just do the same: the root protected init file for your app likely has a mechanism for passing an env var to the new process. Systemd and supervisor do. Then pop os.environ instead of reading it, and you are safe from bots and script kiddies, which is likely your threat model
                                                                                                            • e12e 12 days ago
                                                                                                              If you use bitwarden - you can set them via bitwarden cli, eg:

                                                                                                                  export DATABASE_URL="$(bw get password my_app/db_url/prod)"
                                                                                                              
                                                                                                              And put that in a startup script (that can go in version control).
                                                                                                              • Izkata 12 days ago
                                                                                                                This has the same problem that OP refers to with regards to DEBUG: It remains in the environ, and for example if you forgot to set DEBUG=False, Django will just dump the whole environ on the error page.

                                                                                                                If you can call bitwarden from within the python code somehow, such as with subprocess, that would be better.

                                                                                                                • thaumasiotes 12 days ago
                                                                                                                  Where is bitwarden getting the password from?
                                                                                                                  • e12e 11 days ago
                                                                                                                    This is only a solution for inputting the password. Generally you would set it as a variable on the server (eg: heroku sets this) - but you could provide it on startup for local development/testing.
                                                                                                                    • BiteCode_dev 11 days ago
                                                                                                                      And what happens on reboot ? Then you get to hardware solutions, and a small website is not going to want that.
                                                                                                                      • e12e 11 days ago
                                                                                                                        You need to unlock bitwarden. As mentioned above - this is more a solution for local development (against a non-local db).

                                                                                                                        For a service, one would need some kind of secret management - but a shell script could set this variable/value for input to eg docker or a k8s setup. Or the service (eg: heroku) could handle it.

                                                                                                                        • thaumasiotes 11 days ago
                                                                                                                          "What happens on reboot" is a good question for a scheme that involves not using the filesystem, but bitwarden is not such a scheme. Everything will be fine after a reboot.
                                                                                                                          • BiteCode_dev 11 days ago
                                                                                                                            You need your app to have the bitwarden decryption passphrase to read it. Do you intent to enter that manually if the server reboots ?

                                                                                                                            If not, then it's a secret you must manage, and you are back to square one: either using a simpler solution, use a hardware key, or going full secure vault service, with privileged first request, and so on.

                                                                                                                            • thaumasiotes 11 days ago
                                                                                                                              The passphrase is hardcoded and kept in version control. ;D
                                                                                                                  • BiteCode_dev 11 days ago
                                                                                                                    Env vars in systemd or supervisor files are fine for small server projects. But make sure you pop them from the os.environ dict once you read them to avoid accidental exposure.

                                                                                                                    For the desktop, use the keyring module.

                                                                                                                    If you start to scale up your threat model, you should use a vault, but the setup is way more costly, and tricky to get the reboot story right (hence the priviledged first requests comments).

                                                                                                                    • abdusco 11 days ago
                                                                                                                      > But make sure you pop them from the os.environ dict once you read them to avoid accidental exposure.

                                                                                                                      I've always accepted this was an attack vector, and some malicious library could extract env vars.

                                                                                                                      This actually makes a lot of sense, thanks!

                                                                                                                    • javajosh 12 days ago
                                                                                                                      Theoretically you could write a process that gets its secret from the first request, which is treated as privileged. Then the app can have it in memory, and it's only as vulnerable as the app itself. (This is a kind of degenerate form of the "stem cell" pattern for processes, like Erlang's gen_server.) And yes, this kicks the can to another system, but presumably the secret has to be durable somewhere. Although perhaps a sufficiently sophisticated (or broken?) cluster the nodes could keep passing the secret to each other, never touching disk.
                                                                                                                      • bm-rf 12 days ago
                                                                                                                        I’ve heard that tmpfs is a good solution. A script on boot loads the secrets from something like vault. Env can be exposed through docker inspect if you’re on a shared host. Not sure what other negatives to env there are though.
                                                                                                                    • Gordonjcp 11 days ago
                                                                                                                      Things I do every time I start a Django project:

                                                                                                                      1) bring up Django project

                                                                                                                      2) populate db with example data

                                                                                                                      3) "Wow auto admin is so cool!

                                                                                                                      4) Right anyway, let's get to work

                                                                                                                      5) mkdir my_real_project && cd !! && pip install flask

                                                                                                                      • TedShiller 11 days ago
                                                                                                                        The first thing I do when I start a Django project is to not use Django.
                                                                                                                        • jeremycarter 11 days ago
                                                                                                                          Which is good. Imagine trying to scale a Django app when more than just 5 concurrent users are hitting it. Step 1, don't use Python.
                                                                                                                        • Jaxtek 11 days ago
                                                                                                                          What else do you use then?
                                                                                                                        • hdlothia 12 days ago
                                                                                                                          Good advice
                                                                                                                          • guender 11 days ago
                                                                                                                            • NaturalPhallacy 12 days ago
                                                                                                                              I found this a lot more interesting than the submission article: https://12factor.net/