Doing Things the ‘Right’ Way and Learning Frustration

I always tend to get hung up on this when making my own projects. In my experience it’s usually much easier to do things properly at the beginning, than go back and fix everything you were doing wrong later. It’s also hard to change when you’ve fallen into a bad habit. But when so many people have different ideas of what the ‘right’ way is, it can be really difficult to know for sure.

For example, while doing some reading after finishing the Django Girls tutorial, I found that many people thought it was better to use class-based views. The tutorial did not have us writing class-based views. The more I read, the more I felt that was how it should be done. So I went to my code and changed about half the views to use classes. The others would have been more difficult to convert. And I realized that, although class-based views can definitely be more efficient if you’re doing something simple, they shouldn’t necessarily be used for everything. Sometimes writing your own function is better.

I hadn’t changed anything on my Django blog project for a few days because I wanted to add categories and tags, and I wasn’t sure what the best way was to implement them. I ended up following the example set by the Mozilla tutorial, which has you create a book catalog for a library, and added a Category model which was also a ManyToManyField in the Post model. This is the same way they implemented a genre for books and I figured they were fields that worked the same way – a book can have multiple genres and a blog post can have multiple categories.

I then ran into the problem of how I was going to display a list of categories on every page. I wanted to add it to my base template, but I couldn’t think of a simple way to do that. It turns out that writing a context processor was the answer. All I had to do is make a file called ‘context_processors.py’ (or maybe whatever you want, but this is the usual name) and enter this:

from .models import Category

def categories_processor(request):
    categories = Category.objects.all()
    return {'categories': categories}

This allowed me to easily add categories to my base template with a simple for loop to display the name:

<h3>Categories</h3>
    <ul>
        {% for category in categories %}
        <li> {{ category }} </li>
        {% endfor %}
    </ul>

However, I then had the issue of not knowing how I could make those categories have a count next to them to show how many posts fall into that category. I also didn’t know how I could link to a filtered post list when you clicked a category’s name. I mean, I had an idea of how it could work, but it would involve making a new view and url for each filter (there’d be a tag one soon too), and that just didn’t feel right to me. What I want is like, one view, that takes a parameter that is the filter, and then displays the posts according to that filter, using the same url and the same template. But I couldn’t seem to work out how that should be done.

I did some research, and maybe django-filter is my best option, but I’m just not really clear on how it works. I thought that maybe I should do a bit more basic Django practice since there’s still a lot that I don’t know. So far, all I’ve done are the Mozilla and Django Girls tutorials. While the Django Girls one was good, it was very basic. And the Mozilla one was more complex, but I actually did that one first, and I think a lot of it went over my head. They often told you to just ‘copy and paste this’ – though I very rarely do that when following tutorials as I learn better from typing myself – and I don’t feel they explained enough. My feeling is that when you are teaching someone something that they know nothing about, you should explain as much as possible, especially when it’s something as big as Django. I was going to use the word complex there but I’m not sure if that’s accurate, I mean I think that once you know what you’re doing you can probably use it to make websites pretty easily. But there’s a lot going on there and I wish there were more tutorials written like the Django Girls one where things are broken down in a very simple way.

So anyway, I think I’m putting my blog project on hold until I understand more about what I’m doing. I’m going to complete the official Django tutorial and then watch some Youtube tutorials that have been on my list for a while now.

I know people often say that you shouldn’t get stuck in a cycle of completing beginner tutorials over and over, but for me personally, I feel that by creating a project, even when I’m following along with a tutorial, I learn a lot. Anything that I already know is reinforced, and it also shows the different ways people do things when it comes to coding. Which brings me back to my initial point – I want to do things the right way, and the more exposure I get to different methods, the more likely I am to figure out what that is.

In addition to this I’ll continue to do coding challenges every day, which I have been for the past month or so. I also saw that apparently The Odin Project has just released a JavaScript full stack course, so I’m going to check that out. Previously I hadn’t bothered much with their course because it initially only taught Ruby on Rails, and I was focusing on Python. While I’m not a huge JavaScript fan, I know it’s useful, but what I know is fairly basic. The job I hope to apply for soon will require it so it would probably be a good idea to work on those skills.

That’s the other thing – I’m not sure I’d be using Django at all if I get this job. I just started it and liked it. I frustratingly don’t have all the details, but I was told I have to be strong in Python and have a passing knowledge of Node.js. This is pretty vague to me but it leads me to believe that they use some Node framework like perhaps Express on the backend and use Python alongside it for scripting, or something. I wish I knew more, since it would be much easier to study if I did.

Oh well, I’m going to go and hopefully learn something now.

Static Files in Django

So if you’ve read my earlier post, you’ll know that I’ve had problems with my static files before. But I thought my troubles were over. Then today, I decided to add a few things to my Django blog project, went into the admin to test some stuff and there was no CSS! My blog itself still had the CSS I wrote, but the admin had none. I then went and quickly looked at the live version, but that was fine. I’m honestly not sure what I’ve done that could have affected the admin CSS not showing, but I guess it was something. So, it was Google time!

I first browsed a few Stack Overflow posts, but none were particularly helpful. I usually find all kinds of useful information there, but it wasn’t working out for me today. Then I found this: Django Static Files.

I’d read about the collectstatic command before but not in any context that made it seem like it would help me. But it explains a lot.

So I ran this from my project folder:

manage.py collectstatic

I was informed that this would overwrite existing files but I have version control so I said ‘yes’! Of course, it didn’t really overwrite my files anyway since it was just adding Django’s admin folder. This solved my problem, but I was still curious about what exactly I was missing about static files to begin with.

I read Django’s official documentation on the matter here: https://docs.djangoproject.com/en/2.2/howto/static-files/

I thought I looked at that when I was having trouble with uploading my static files to Heroku, and maybe I did, but didn’t know enough to understand what I was reading, or something. It happens.

So I tried to adjust my static file situation to follow Django’s instructions. I’d originally had my static folder within my ‘blog’ folder, ‘blog’ being a Django app. That was how Django Girls had me set it up. I guess if I’d run collectstatic it would have put those folders in a static folder in my main directory, so that Heroku would have seen it. I’d ended up making that folder manually in order for my deployment to work. But Django Girls never mentioned using collectstatic. They also had you putting your CSS folder directly in the static folder, while Django recommends using appname/static/appname/css/style.css so that if you have multiple apps with static folders, and some have CSS files of the same name, you won’t run into the problem of only one of them working. So I adjusted my folder structure accordingly.

I then ran collectstatic again, and my newly created blog/static/blog/ folder with all my blog app’s static files was added to the base static folder. I deleted my manually added files, which were in the static/css folder. This broke my CSS but I was expecting that. I had to go to my style declaration to specify that my CSS file was now in blog/css/. This is what that line looks like now:

<link rel="stylesheet" href="{% static 'blog/css/style.css' %}">

I then edited my Procfile so that Heroku would run collectstatic:

web: python manage.py collectstatic --noinput; gunicorn myblogsite.wsgi --log-file -

Now was the real test – deployment! I was scared. But it worked, sort of. Heroku did indeed appear to collect a bunch of static files, and my live site wasn’t completely broken. But it didn’t have CSS. Only the admin side did.

I found out that I can access my files on Heroku by going to the terminal and running:

heroku run bash
ls
cd static

The first line is to connect, the second line will list all files in the base directory, and the third changes to my static file directory. If you’re reading this I’ll assume you have at least a basic understanding of how to use the terminal and leave the explanation at that.

My project files such as manage.py were there in the base directory, as well as my static folder. I went there and checked it out and sure enough, the only folder that was inside was the admin folder. So for whatever reason, my blog files weren’t being collected even though they were when I ran collectstatic on my local project. I even double checked by deleting all the collected files from my blog app and rerunning the command. Yep, they got created. So why doesn’t it work on Heroku?

So it turns out I’m dumb. I had gone and added my static folder to .gitignore now that I knew I was supposed to be building that folder on my server (and it was in the original .gitignore file that Django Girls had me create). But I’d written ‘static/’ as opposed to ‘/static/’ so that was making it ignore all static folders in my project, instead of just the static folder in my project directory root. So Heroku didn’t have my blog/static files at all. When I changed it to ‘/static/’ my blog folder’s static files were uploaded properly so Heroku was able to find them when it collected my static files. And then, yay, CSS on my live website!

Apparently Heroku is supposed to run collectstatic without having to tell it to, but I’m done messing with it for the moment. I’d like to get some real updates on my project so I want to try not to break anything else today. Well, at least not anything else involving static files.

Key takeaways from this experience:

  • Read the docs! I need to get better at that. Honestly, especially when I was just starting, I often found the docs confused me more than they helped. But now I’m starting to understand them better and I realize just how important it can be to read them properly.
  • Double-check your .gitignore. Additionally, your code editor may have a simple way to see which folders and files are ignored. I use Visual Studio Code, and all my ignored stuff is grayed out in my file explorer.
  • Make sure you know what tools are available to you for debugging. It was a big help to me knowing how I could access my Heroku files, so that I could see for sure that my static files just didn’t exist there.

Deploying a Django Site to Heroku

I recently deployed a Django website to Heroku for the very first time. It’s a blog, but it’s more of a project for my portfolio than a real blog I intend to maintain. If I ever decide to go to that much effort for one site for myself it will be for a browser-based game like Kingdom of Loathing.

Anyway, I was finishing up the Django Girls tutorial where we deploy our finished website to Heroku, and I ran into some problems. I found it frustrating trying to find how to fix everything that was wrong, and when I finally did I decided to document my experience in case anyone else can benefit.

My blog has PostgreSQL as the database which I understand is the default on Heroku. Your experience may differ if you’re using a different database. Also, disclaimer: I’m not an expert, but this worked for me and I’ll explain as well as I can.

I’m not going to tell you everything from the beginning because it’s already been done so well on the Django Girls tutorial. You should go there first and follow the instructions, but come back here before deploying, because it will not work.

One important thing I’d like to mention is that you shouldn’t put your database password or your SECRET_KEY straight into your settings.py file. This won’t necessarily hurt the functionality of your site but it’s bad practice as they aren’t protected. Instead you should use environmental variables. I’ll probably post about them in more detail at a later date. For now, just know that you can install python-dotenv and adjust your settings.py like this:

from dotenv import load_dotenv

load dotenv()

...

SECRET_KEY = os.getenv('SECRET_KEY')

Write your database password in the same way as the SECRET_KEY. Then, make a file called .env in the same folder as your settings.py file. In this file enter your secrets:

SECRET_KEY=yoursecretkeyhere
DATABASE_PASSWORD=yourdatabasepassword

Keep in mind that your variable names have to match. You could technically call these whatever you want but remember that they are variables and have to be referenced by what you write in this file.

Your site should now run locally, but you still have to share your secrets with Heroku. Luckily this is really easy. Just go to your Heroku dashboard, select your app, go to settings and hit the ‘Reveal config vars’ button. Here is where you put in your secrets.

Now you have to solve a problem. Try and deploy your Heroku app now if you want, but it probably won’t work because it won’t be able to collect your static files. Remember when Django Girls had you enter this in your wsgi.py file?:

from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(application)

Go ahead and delete those lines. That information is outdated. Instead, you do this in your settings.py file:

MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
...

The new line is the ‘whitenoise’ one, but I included the others so you can see where it has to be positioned. Right after the security line and before the others. That’s it. This made my static files stop working, but I fixed it by moving the folder from my blog app folder to my base project folder – whether that is ‘best practice’ or not, I do not know, but it’s what worked for me. At this point I thought I’d fixed my problem and went to try deploying again, and again, it did not work. My research took me to Heroku’s deployment guide for Django apps. After doing what we’ve just done with whitenoise, they want you to put this in your settings.py file:

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.9/howto/static-files/
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = '/static/'

# Extra places for collectstatic to find static files.
STATICFILES_DIRS = (
os.path.join(BASE_DIR, 'static'),
)

This broke my static files again locally. My static folder is called ‘static’, by the way. To fix my static files I had to make STATIC_ROOT point to ‘static’ and STATICFILES_DIRS point to ‘staticfiles’ (I got errors when I tried making them both point to ‘static’. I tried deploying to Heroku again, and again, it did not work. In a whirl of frustration I started trying random things and my deploy finally worked when I simply commented out the STATICFILES_DIRS line.

At that point I was super excited, but my live site didn’t work – I was getting a 500 Server Error page. I realised it was database related, since I had a static page which worked just fine and could also get into the admin panel. I made my superuser and entered the admin console, but couldn’t load anything in the database. So, back to Google. Someone said that running manage.py makemigrations and migrate on the server was the solution. It was not. Then, someone said Heroku needed the migrations folder. Now, I had added that to .gitignore, since I had read that you could get conflicts if there were different migrations folders being merged into a repository. I also had the impression you just made the migrations again on the server after deploying. I always try to do what is considered best practice, but apparently that information was wrong. I unignored my migrations folder, pushed to Heroku again, and did ‘heroku run python manage.py migrate’ to…. migrate the migrations! Well, that worked.

I’m going to mention here that I realise the Django Girls site has you run migrate once your site is up. But at this point it had been like three hours since I’d left the tutorial, and I’d forgotten all about it.

Now you’ve hopefully got your site up and running! It doesn’t matter how simple it is; I feel getting it deployed is a huge step after the ordeal it was for me and it’s something you can be proud of.