Takeaways

  • Experience writing and using decorators
  • Web app basics (server-side programming)
  • Encoding (serializing) & decoding (deserializing) JSON data

Relevant Resources

Inator Search-inator

New employee orientation was awkward and horrible. Don’t feel bad. No -inator could have avoided that.

Our Story Continues

Doofenshmirtz barges into your office. “Since we hired these temps, the number of inator-related incidents has gone up dramatically!” He pulls a flip chart out of nowhere and starts pointing at pages. One is a graph with a big red line trending upward. The next is blue, trending downward. The next is a picture of a puppy.

“You get the idea” he says.

“I think part of the problem is that no one here knows which of these inators actually works anymore. Kevin got explodinated after he pulled the lever on the explodinator. To be fair, it does look a lot less dangerous than it is.”

“You’re a hip… computer person. Why don’t you whip up one of those fancy… oh… what’s it called… internet-inators. Yeah, make one of those web-inators, so the temps can enter some information about the rest of the inators we’ve got. One more accident and OSHA is going to send out their inspectors again. You know, it does seem a little weird that they don’t seem to have a problem with all the evildoing as long as everyone wears closed toed shoes.”

You then begin an unexpected long and thoughtful discourse with your new boss about whether or not overalls are coming back in style.

Your Task

Your task is to develop a web application that can be used by Dr. Doofenshmirtz’s temporary employees to track the locations and conditions of his inators2. You are to do the following:

  • Complete the partially implemented program, so that your web application behaves as described in this post and the searchinator documentation.
    • Implement the handlers in searchinator.py
    • Implement the functions marked with TODO items in utils.py

Refer to this assignment’s rubric for further description of point values for completed tasks.

Application Description

The purpose of the web application is to allow Dr. Doofenshmirtz’s helpers to…

  • … list all of the inators currently in the inventory.
  • … view details for a specific inator.
  • … add new inators to the inventory.
  • … delete an inator from the inventory.

To make sure that only authorized employees are able to use the web application, it will require users to login to access most pages.

Program Design

The web application (“the app”) will respond to the URL paths listed below. If the app receives a request for a path that is not listed below, it should respond with an HTTP 404 Not Found. Fortunately, Flask will handle that part for you automatically! If your app receives a request for an unknown path, Flask will respond with a 404.

The following list describes how the app should respond to requests for each path. Note that certain paths should be capable of handling multiple HTTP request methods (e.g., GET and POST). Further description of the request handlers and helper functions can be found in the documention.

  • /
    • GET Request
      • List inators sorted first by name and category
        • ACTUALLY_WORKS inators at the top, HOPELESSLY_BROKEN inators at the bottom
        • Sorted alphabetically in each category
      • Requires users to be logged in to view the page
      • Handled by searchinator.list_inators(data)
  • /add/
    • GET Request
      • Shows a form for a user to add a new inator to the inventory
      • The form has the following fields. All fields are required.
        • name - a string
        • location - a string
        • condition - a dropdown menu with 5 options
        • description - a string
      • Requires users to be logged in to view the page
      • Handled by searchinator.add_inator(data)
    • POST Request
      • Accepts form data submitted when a user fills out a form for a new inator
      • Requires users to be logged in to process the form
      • Handled by searchinator.add_inator(data)
  • /view/<ident>/
  • /delete/<ident>/
    • GET Request
      • Shows a form for a user to confirm the deletion of an existing message
      • This form has no fields; it only has a submit button
      • Requires users to be logged in to view the page
      • Handled by searchinator.delete_inator(data, ident)
    • POST Request
  • /login/
    • GET Request
      • Shows a form for a user to enter login credentials
      • This form has the following fields:
        • username - a string
        • password - a string
      • Users are NOT required to be logged in to view this page
      • Handled by searchinator.login(data)
    • POST Request
      • Accepts the form submitted when a user fills out the login form
      • Users are NOT required to be logged in to process the form
      • Handled by searchinator.login(data)
  • /logout/
    • GET Request
      • Shows a form for a user to confirm that they want to log out
      • This form has no fields; it only has a submit button
      • Users are NOT required to be logged in to view this page
      • Requires users to be logged in to view the page
      • Handled by searchinator.logout()
    • POST Request
      • Accepts the form submitted when a user fills out the login form
      • Users are NOT required to be logged in to process the form
      • Requires users to be logged in to process the form
      • Handled by searchinator.logout()

Error Handling

Refer to the documentation to discern how to handle errors. When necessary, functions should raise errors using the abort() function.

If you have questions about what should happen in specific cases or questions about assumptions you can make, ask your instructors for clarification.

Working on the Web Application

Setting up a Virtual Environment

You are going to need a virtual environment for this assignment.

# After you've cloned your starter code and changed into your project directory...
$ virtualenv --python=$(which python3.5) env

# Whenever you want to work on your project, you have to source env/bin/activate
$ source ./env/bin/activate

# You know you've activated your virtual environment if you see "(env)" in front
# of your shell prompt!
(env) $  # wow!

The above command will create a virtual environment configured to use Python 3.5. All relevant files will be stored in a new subdirectory called env which you should not track with git. We don’t want to see any virtual environments in GitLab.

  • For sanity’s sake, just name your virtual environment env as shown above.
    • The starter code includes a .gitignore that tells git to ignore the directory named env/, so you won’t have to worry about accidentally committing it.
    • We’ve included a configuration file for pytest (pytest.ini), so that it will ignore env/ as well. This prevents pytest from exploring those packages for more and more unit tests to run.
  • Be sure to install the packages listed in requirements.txt.

    $ source env/bin/activate
    (env) $ pip install -r requirements.txt
    

Running it

For this assignment, your program will be a web application. It accepts two optional command line flags that allow the user to specify the host and port number that the server will bind to.

  • --host: Tells your Flask server which address it should use to listen for incoming HTTP requests.
    • 127.0.0.1 is your loopback address. You can only access it if you’re using the machine the server is running on.
    • 0.0.0.0 is the “anywhere and everywhere” address. This tells the server to listen to HTTP requests coming in through any network interface.
  • --port: Tells your Flask server which port number to use when listening on --host.
    • 5000 is the default.
    • Really anything above 5000 is probably fine. You won’t have permission to use low port numbers (less than 100), so don’t do that. Stick with numbers in the 5000 to 9000 range. If a port is taken, just pick another one.
# Assuming that we've activated our virtual environment, and we've
# installed the packages from requirements.txt...

# We can use --help to see how to use "flask run"
(env) $ FLASK_APP=searchinator.py FLASK_DEBUG=1 flask run --help
Usage: flask run [OPTIONS]

  Runs a local development server for the Flask application.

...

Options:
  -h, --host TEXT                 The interface to bind to.
  -p, --port INTEGER              The port to bind to.

...

We can start the application like so:

# Assuming that we've activated our virtual environment, and we've
# installed the packages from requirements.txt...
(env) $ FLASK_APP=searchinator.py FLASK_DEBUG=1 flask run --host=0.0.0.0 --port=5050
 * Serving Flask app "searchinator"
 * Running on http://0.0.0.0:5050/ (Press CTRL+C to quit)

You can now interact with the web application by pointing a browser to http://<hostname>:<portnum>/. The value of <hostname> depends on where you are running the server. The value of <portnum> is whatever you specified with the --port flag. For example…

  • Running the web app on your local machine:
    • Your hostname will be localhost.
    • Let’s say you used --port=8034. In that case, your port will be 8034.
    • Point your browser to http://localhost:8034/.
  • Running the web app on a campus machine:
    • Let’s say you’re using rc06ucs213.managed.mst.edu. Your hostname will be rc06ucs213.managed.mst.edu.
    • Let’s say you used --port=9255. In that case, your port will be 9255.
      • Point your browser to http://rc06ucs213.managed.mst.edu:9255/.

Using campus machines

If you’re working with a campus machine, use one of the rc__ucs213.managed.mst.edu machines. Note the u in there.

That is,

  • Walk to the CS building
  • Walk into room 213
    • It is the one with the couches and the arcade machine.
  • Sit in front of a Linux box
  • Use the Linux box that’s in front of you

or

  • SSH into one of those boxes

Available ports

If you’re using a campus machine to work on this assignment, there is a non-zero[^favorite] probability that someone else will be working on the same assignment on the same computer.

So?

So, if you both try to bind to the same port number (let’s say port 8080), then one of you is going to see this message:

(env) $ FLASK_APP=searchinator.py FLASK_DEBUG=1 flask run --host=0.0.0.0 --port=5050
 * Serving Flask app "searchinator"
Traceback (most recent call last):

    Hooooooly trackeback, Batman.

OSError: [Errno 48] Address already in use

In that case, just pick a different port number

# Assuming no one else has taken THIS port....
(env) $ FLASK_APP=searchinator.py FLASK_DEBUG=1 flask run --host=0.0.0.0 --port=5050
 * Serving Flask app "searchinator"
 * Running on http://0.0.0.0:5051/  (Press CTRL+C to quit)

Be careful that you’re using the correct port number when you connect to it with your browser.

Multiple PuTTy Sessions

If you are going to edit code with PuTTy, then listen up. While the application is running, the PuTTy session you have opened will be busy. This means you will not be able to navigate to files or open them while your Flask app is running. The simple solution is to open up a second PuTTy session and connect to the same machine. The second PuTTy session will be where you do your work, and the first one will run your application.

Tips for running the app

  • You can only interact with the web app (load pages) if it is running.
  • You can kill the web app with Ctrl-C.
  • If you modify one of the Python files that comprises the web app, it will automatically reload those changes.
  • If you modify one of the Python files that comprises the web app and it causes an error (such as a syntax error), the web application program will die, and you will have to restart it. You may have to choose a different port if Python does not clean up after itself when it dies.

Tips for interacting with the app

  • All paths must end with /. /add/ is not the same as /add. Your app expects all paths to end with /.
  • Any ol’ browser should do. Firefox is good.
  • Make sure you’re using the right port number when you try to pull up your app.

Automated Tests

The starter code includes a complete (enough) test suite. You are welcome to add additional tests, but the current test suite should cover all the bases. DO NOT modify the existing test suite without the advice of your instructor.

Note: passing the test suite does not guarantee that your code is correct. You should use them to help develop the project, but you need to pay careful attention to your implementation as well.

Make sure your app looks right, too. Check that your list is the right color, messages are flashing properly, etc.

Running the tests

This time around, you will be the one installing pytest. You should use a virtualenv to create an isolated environment to use with this project. Note that the requirements.txt file lists pytest as a dependency. If you use requirements.txt to set up your virtualenv, then you’ll have the right version of pytest installed.

# Assuming that we've activated our virtualenv (called env in this
# case) and we've installed the packages listed in requirements.txt
(env) $ py.test

Note that you don’t need the ./ this time around. Activating the virtualenv adds flake8 to your PATH environment variable, which is nice.

requirement.txt specifies the expected version of pytest. This is the exact version that we will use to grade your assignment.

A couple more notes:

  • The starter code includes a short configuration file named pytest.ini. That file prevents py.test from exploring your virtualenv for more tests to run. You only need to test your code. You don’t want to run all the unit tests for all the third-party libraries that were installed…
  • The starter code also contains some testing helper functions in conftest.py that you can look at. pytest fixtures are neat.

Proper Python Style

Like py.test, you’re on your own to install flake8 in your virtual environment.

# Assuming that we've activated our virtualenv (called env in this
# case) and we've installed the packages listed in requirements.txt
(env) $ flake8 *.py

Make sure that flake8 has no output when you pass it your files. All of the .py files (including tests) should pass flake8’s checks. If there are any messages (warnings, errors, etc), you should address them. Also, do not suppress or exclude any of flake8’s rules.

The requirements.txt file specifies the following version of flake8, which is exactly what we will use to grade your assignment.

Further Details

Important Notes

  • The authentication in this app is performed very simply by storing a value in the user’s session (key: username, value: logged in user’s username).
  • Do not rename functions, follow the documentation EXACTLY.
  • You should tackle this assignment as follows - seriously do this so you’re not overwhelmed:
    1. Read this entire posting and docs (you’ll likely do this more than once)
    2. Set up your virtual environment
    3. Install the packages from requirements.txt with pip
    4. Make sure the web app runs and you can access it from your browser
    5. Implement the incomplete utility functions in utils.py
      • Verify things are working by using the unit tests
    6. Implement handlers in searchinator.py starting with list_inators and view_inator
      • Don’t forget to apply decorators as necessary!
      • Generate some testing data using generate.py
      • Verify things are working by using the unit tests
    7. Finish implementing the rest of the missing functionality
    8. Verify that all tests pass on a campus machine
    9. Fix any style issues
    10. Submit

Tips and Hints

  • Make sure your virtual environment is using Python 3.5. Otherwise, you may get weird results.

  • Be mindful of the order in which you apply decorators. The order is important.

  • You may find it easier to run the tests from one, specific test file. You can do that like so:

    (env) $ pytest test/test_utils.py
    
  • Update your success/warning/error flash messages to match the messages expected by the unit tests. Do not update the unit tests to match your messages.

  • Do not modify the existing unit tests. You may add your own, but the instructor tests (mentioned in the rubric) will include the exact tests that came with the starter code. If you have questions about the current tests, email your instructor.

  • You are welcome to add your own unit tests! Feel free to add more test_*.py files to the test directory to help you develop your web app. Just make sure your code passes the provided tests AND your own.

  • Don’t waste a bunch of time copying docstrings this time. A brief, single-line docstring is sufficient this time around.

Point Value

This assignment is worth 100 points. It will be graded according to the following rubric:

Feature Points Possible Mostly or completely incorrect (0% of points possible) Needs improvement (50% of points possible) Adequate, but still some deficiencies (75% of points possible) Mostly or completely correct (100% of points possible)
Unit Testing          
Student passes instructor tests 20 -20 -10 -5 0
Implementation          
Demonstrates understanding of Flask (request.form, abort, flash, redirect, session, etc.) 10 -10 -5 -3 0
Demonstrates understanding of decorators 20 -20 -10 -5 0
Demonstrates understanding of json module 5 -5 -3 -2 0
Demonstrates understanding of exception handling 5 -5 -3 -2 0
Demonstrates understanding of context managers and file IO 10 -10 -5 -3 0
Functionality          
Program runs without runtime/syntax errors 10 -10 0
Style          
flake8 outputs no suggestions for improvement for any “.py” file (no warnings, no errors, etc.) 10 -10 0
Code Review (sufficient comments, implementation looks Pythonic, etc.) 10 -10 -5 -3 0
Points possible 100        

Each cell indicates how many points out of the available points will be awarded for that feature (row) and assessment level (column).

Cells marked -- cannot be achieved.

Submission

Deadline

  • 11:59:59 PM CDT – Monday, October 9, 2017

Submission Procedure

Refer to the assignment submission page on the course website for details on submitting your code to GitLab.

Grading Procedure

When your assignment is graded, we will do the following (roughly).

# Set up and activate our own virtualenv
$ virtualenv --python=$(which python3.5) env
$ source env/bin/activate
(env) $ pip install -r requirements.txt

# Run flake8 and check for style issues. Note versions in the requirements.txt.
(env) $ flake8 *.py test/*.py

# Run unit tests. Also note versions in the requirements.txt.
(env) $ py.test

# Run your main file and check the output
(env) $ FLASK_APP=searchinator.py FLASK_DEBUG=1 flask run

# Then we're going to see if it works correctly.

FAQ

Now that I’ve gotten into this virtual environment, how do I get out? Am I trapped here forever? Who’s going to take care of my dog?

When you log out, your virtual environment will go away. Each time you log in, you’ll have to reactivate it.

If you’d like to deactivate the virtual environment without logging out, use the deactivate command. After running deactivate, you’ll notice your little (env) indicator disappears.

You can always reactivate with source env/bin/activate later.


Everyone is trying to use the same machine, so it’s slow, and I am feeling very salty and unpleasant about that.

First, write your congressman about funding issues.

Then consider using an SSH tunnel.

Basically, what we want to do is use Putty to create a tunnel3 that forwards your traffic to the Linux machine you’re using. This way, your browser can talk to a port on the local Windows computer you’re using, and PuTTY will forward that traffic to the remote Linux machine you’re coding on. If you haven’t realized already, SSH (and PuTTY, I guess) is an incredibly useful tool. You can do a lot more than just run a shell.

So here’s a quick-and-dirty tutorial:

  1. If you’re off campus, you’ll need to connect to the campus VPN.
  2. Open PuTTy.
  3. Load up a host (the xcs machines work too!), but DON’T open a connection yet.
  4. In the Category pane on the left: a. Open the Connection dropdown b. Open the SSH dropdown c. Click the Tunnels category. You’ll see some checkboxes and a form.
  5. In Source port enter your favorite port number. You’re going to run your Flask app on that port later, so pick a sturdy one.
  6. For the destination, enter localhost:<portnum> where <portnum> is the port number you chose in the previous step.
  7. Press the Add button.
  8. Click the Open button to start the SSH session.
  9. Refer to the earlier parts of this post that explain how to get your Flask app running.
    • You won’t need the --host option, since we want the default (127.0.0.1).
    • You will need the --port option. The port your app runs on must match the port number you chose above.
  10. Once your Flask app is running on your favorite port, point your browser to localhost:<portnum>, where <portnum> is your chosen port number.

If you need to run your app on another port, you’ll have to start over and configure your tunnel differently.


Do users need to be logged in to view /logout/?

Yes.

The Program Design section of this post was incorrect, but it’s been updated. The module docs and unit tests are correct.


Is it OK for the user to submit blank values for /add/?

If the user doesn’t enter a value for name, location, or description, that’s fine. Those are allowed to be blank4. There’s no need to do anything special if any of those fields are blank.

However, if condition is blank5, we have a problem. As the docs explain, we have to be able to construct a condition.Condition object out of the value we receive. The empty string can’t be converted to a Condition, so we’ll abort(400) for that.

  1. It’s close, but it’s not quite what we need. 

  2. Amazingly6, this tool is easier to use, faster, and more reliable than any Peoplesoft product to date. 

  3. An imaginary tunnel, tbh. 

  4. Even though blank fields are not very useful for searchinating. 

  5. Yes, they’re choosing a condition from a drop down. In reality, it’s not likely they’d send a blank form. Regardless, you still need to handle that case. 

  6. I guess it’s not that amazing. We’re setting the bar pretty low for that comparison.