git - exclude files in a diff

In some scenarios you might have a build server that doesn't have access to the Internet, in which case if you're working with something like npm, you might need to commit your node_modules folder to the git repository to get stuff to work.

Of course, it might be better to setup your own artifact repository such as Nexus on an internal network for these dependencies, BUT, I didn't have that! :D The node_modules are included as part of a commit.

Having to do a code review of a pull request can be tricky because of this, due to more than 10,000 files being in the commit, of which only 40 are actual source files of interest. We're using TFS 2017 which has a nice web interface for doing this, however it only supports showing 1000 files at a time.

So back to the best way to use git: the command line :)

To exclude the node_modules from the diff was as simple as doing the below:

$ git diff master develop -- . ':!node_modules'

Of course, I used difftool because I prefer WinMerge but it works the same.

Finding currently executing queries and parameters

If you've worked with an Object Relational Mapper (ORM) such as Hibernate, NHibernate, Entity Framework, etc., then you'll know these sometimes convert your queries into parameterized queries that can become quite beastly.

Trying to debug it means either profiling the database to see what queries are executing, and then seeing the query and values from there, or if you can't profile it, maybe selecting from some system tables to do a similar thing.

This snippet below (thanks again StackOverflow) will give you some nice details about the queries on a database, including an XML query plan that contains the parameters that were used when generating the query plan. It can be useful in diagnosing slow queries:

select * 
from sys.dm_exec_requests r 
cross apply sys.dm_exec_query_plan(plan_handle) as qp
cross apply sys.dm_exec_sql_text(r.sql_handle) 
where r.database_id = DB_ID('<dbname>') 

tail on Windows with PowerShell

One of the well known Unix/Linux commands is tail, which gets the tail of a file, meaning the end of a file and prints it out to the output stream.

I used to try find nice programs for this, including Baretail which has some very nice features like colouring lines in that match certain patterns etc.

But if you just want a simple, and now built into Windows solution, just use PowerShell:

Get-Content -Tail 10 filename.txt

This will show the last 10 lines of the file. You can even follow the tail, or watch it for changes, as below:

gc -Tail 10 -Wait filename.txt

Another nice thing you can do is then pipe this into Select-String to filter the output:

gc -Tail 10 -Wait filename.txt | Select-String -Pattern somepattern

diff PDF files

At a client we have a PDF template that needs to be used for registering users. When this is updated, the template is overriden. But it would be nice to be able to easily see what changed between the two.

Well, this is possible :)

You can use Imagemagick and Ghostscript to do this.

You should be able to just do the below:

magick compare old.pdf new.pdf diff.pdf

But that didn't seem to work well. What seems to work better is to rather convert the PDF to an image first, and then compare each page.

magick convert -density 300 -quality 100 old.pdf old.png
magick convert -density 300 -quality 100 new.pdf new.png

magick compare old-0.png new-0.png diff-0.png

Depending on the options you can get better or worse results. Changing the colorspace to CMYK for example could yield better results, or maybe using options to blur/sharpen/despeckle the image too. Try play around with the options to see better results.

The best results I had was actually via using Adobe Acrobat to first export the PDF to images, and then run the compare. I still need to figure out why, because unfortunately I only have the license at work, so this won't always be an option. I'll hopefully update this post in future to show my favourite variation :)

Modern Database Development Practices

Back in November 2012 I presented on this topic at Entelect's Dev Days.

I've also done a blog post previously on setting up Flyway.

I keep referring back to the same links and concepts and this post is here just for that, as a pointer to good resources and brief points about this topic.

I may revisit it and put down my own consolidated content, but for now this will do.

• Reading
• Evolutionary Database Design
  Martin Fowler, 2003
• Agile Database Techniques: Effective Strategies for the Agile Software Developer
  Scott Ambler, 2003
• Refactoring Databases: Evolutionary Database Design
  Scott Amber and Pramod Sadalage, 2006
• Catalog of Database Refactorings
  Scott Ambler
• Three Rules for Database Work
  K. Scott Allen, 2008
• The Baseline
  K. Scott Allen, 2008
• Change Scripts
  K. Scott Allen, 2008
• Versioning Databases - Views, Stored Procedures, and the Like
  K. Scott Allen, 2008
• Versioning Databases - Branching and Merging
  K. Scott Allen, 2008
• Scott Ambler's Articles
  Scott Ambler
• Stairway to Database Source Control
  SQL Server Central
• Martin Fowler's Practices
• DBAs collaborate closely with developers
• Everybody gets their own database instance
• Developers frequently integrate into a shared master
• A database consists of schema and test data
• All changes are database refactorings
• Automate the refactorings
• Automatically Update all Database Developers
• Clearly separate all database access code
• K. Scott Allen's 3 Rules
• Never use a shared database for development work
• Always have an authoritative source for your schema
• Always version your database
• Tools
• Redgate Developer Tools
• SQL Server Data Tools (SSDT)
• Entity Framework Code First Migrations
• DbUp
• Flyway
• Open DBDiff
• sqlcmd
• tSQLt
• dbdeploy