Working with Jenkins I've been frustrated by the way it uses static checking of python code via pylint. The purpose of a continous integration tool is to fail a build if it doesn't work. However it's process for static checking of Python code is fundamentally flawed, making it essentially useless.

How Jenkins should do Pylint.

When you push a new build to Jenkins, it runs the test suite and fails the build if any of the tests fail. It's simple and precisely the way it should be. If a test has failed then the code doesn't do what the developer expects it to do. Pushing this version to production would be bad and Jenkins fails, e-mails the developers and stops the testing script. So any subsequent commands like deploys, tagging of commits etc. aren't executed.

Great. Sounds good. Pylint should work the same way. If it detects an error in the code, like an AttributeError, unknown name error or similair catastrophic errors, then the build should fail, developers emailed and testing script stopped.

Where Jenkins fails at Pylint.

Instead of this solution. Jenkins instead counts the number of pylint violations. Making no difference between disasterous errors that can crash an application and simple convention violations like number of lines between classes that have an effect on readability but not on the execution of the program.

You can set a limit on the number of pylint violations that can be in an application and fail the build if there's more than that. Which is nice if you're on a team that's highly committed to readability and want to fail if pylint readability decreases.

But replacing a whitespace convention violation with an application crashing error wouldn't fail and that's silly.