Having seen amazing videos of gource I wanted to vizualize the changelog of the codebase I work on. The only trouble was that gource does not understand perforce change log format directly. Fortunately, gource can read text files where each line is a change to one file in the format "timestamp|author|filename|action|colour".
Perforce logs produced by `p4 describe -s` command have the following format:
Perforce logs produced by `p4 describe -s` command have the following format:
Change 106 by max@max_perforce on 2008/10/20 16:17:31 mt makefiles Affected files ... ... //depot/infra/main/src/infra/Makefile#3 edit ... //depot/infra/main/src/infra/Makefile.mt#3 delete
Which should be converted for gource to:
1224515851|max|M|//depot/infra/main/src/infra/Makefile| 1224515851|max|D|//depot/infra/main/src/infra/Makefile.mt|
It is a two-stage process to convert perforce logs to gource logs. The first stage is to extract perforce logs into a file, because it takes a while and you may like to make several passes over it. It can be done with a couple of lines of bash:
$ p4_head=$(p4 changes -m 1 | { read -a words && echo ${words[1]}; })
$ { for((i = 0; ++i <= p4_head;)); do p4 describe -s $i; done } > p4.log
The second stage is to filter and convert perforce changelog into gsource log. Here is how to convert changes only in //depot/infra/main with the python script I wrote:
$ ./p4-gource.py -p //depot/infra/main -o main.gource p4.log
Now gource can be used to vizualize the changelog:
$ gource --highlight-all-users main.gource
Hi Max,
ReplyDeleteWhat is the final output of these scripts? We use perforce too, so i am interested to see if your script would give us something cool.
Cheers,
Ivan
Hi Ivan,
ReplyDeleteScripts input is perforce changelog. The output is the changelog in gource format. The gource changelog is input to gource, which renders an animation visualizing the history of your depot. Have you seen the videos mentioned in the post?
Good post - Helped out a lot. One thing that caused me to stumble (given my lack of Python experience) is that the blog strips out the group name in the regex, so when I simply cut and paste the python code it didn't work.
ReplyDeleteThanks. And yes, I am struggling with formatting of this. It is switching between HTML and WYSIWYG mode in the online editor that breaks my formatting. I should probably try using a decent HTML editor...
ReplyDeleteMaxim, you need to add the perforce 'purge' command, I just mapped it to the gource delete 'D' command.
ReplyDeletep4_action_to_gource = {
"add" : "A"
, "edit" : "M"
, "integrate" : "M"
, "branch" : "A"
, "delete" : "D"
, "purge" : "D"
}
At line 75 the assignment "file=gource_file" is not right, is it? How should it be?
ReplyDelete@Kostas: it is python-3 style print function. http://docs.python.org/library/functions.html#print
ReplyDelete@Lou: nice one, I updated the source.
ReplyDeleteThanks a LOT for this script! It really helped me out!
ReplyDeleteCheers!
Fantastic! Two years later and this post is still incredibly useful. Thanks a lot, and job well done!
ReplyDeleteI use git-p4 to get the same result, just import the part of the depot you want and gource will work out of the box
ReplyDeleteThis was very cool, thanks a lot for this! :)
ReplyDeleteHi there- I am trying to get in touch- I saw you had run into an SSL bug with FTPSSL .. Did you solve it for Python 2.7, and how?
ReplyDeletehttp://bugs.python.org/issue8108
Thanks, I'm working on an open-source Dropbox clone that uses FTP:
https://code.google.com/p/iqbox-ftp/
Hi Max,
ReplyDeleteTrying to use your script resulted in a datestamp error for me.
Debugging it the first regex never matched because our usernames and therefore the use a dot in their names eg. "m.yegorushin" - since your regex uses a \w+ the dot didn't match - changing it to \S+ fixed that problem for me.