PHYSICS SOFTWARE DEVELOPMENT CHEATSHEET

            Greg White, SLAC, Original Version 30-Aug-2019
            Modified GW, 17-May-23 Add getting prev HEAD, so easy compare / undo of file change
                         23-Aug-21 Updated git clone on prod to use GIT ssh support added
                         15-Jan-21 Added how to see what commits you're ahead of origin/master
                         16-Jun-20 Added most basic workflow section
                                                 
Basic References
================

  * /afs/slac/g/cd/swe/git/repos/ - SLAC accelerator sofwtare related repos are hosted here. Nominally these 
    are the golden masters, though for multi-site collaborations these may be downstream clones for GitHub etc.
  * https://lcls-dev3.slac.stanford.edu/git/ - a GitList web homepage for accessing the above repos.  
    
Most Basic Workflow 
===================
Assuming the remote git repo to which you want to contribute already exists, these are the 4 basic commands of git:

    git clone <repo-url>       # Get a remote repository. repo URL may be http or simply a filename. Like cvs checkout
    cd down to repo
    edit files
    git add .                  # Tell git you've made changes to files so it should add those changes to its list
    git commit -m "<comment>"  # Tell git the adds so far make up a self consistent set that should be tracked together
    git push                   # Update the remote repo. Equivalent to cvs commit.

There are examples of the above commands following.
 
Create a "repo" 
===============
Files you create and track with git, are managed as a colection called a "repo" - short for repository. 
So start by making your repository at SLAC. This will be your central reference source code. You'll do 
work later by checking out this repo, and adding files. 

Most of the repos used for accelerator related software at SLAC are collected in AFS 
under /afs/slac/g/cd/swe/git/repos/. So start by finding a place under there where you can create a repo directory 
to host your repo.

Example, for a project to scan matlab files for issues (related to changes for the 2020 CU machine), I found 
there is an existing directory for matlab related repos at /afs/slac.stanford.edu/g/cd/swe/git/repos/slac/matlab. 
So, I create and initialize my repo under there:
 
    cd /afs/slac.stanford.edu/g/cd/swe/git/repos/slac/matlab
    git init --bare matlabscanner.git                # Make our golden master bare repo for the files of my project
 
Note: later, when you first "git push" to a new repo, you may well have to use the long form "git push origin master".

Setting Up for Development with your new repo
=============================================

Make a directory for your local development, if you don't already have one. This will be in your own directory
space - not the shared space as th eabove repo creation. I often put git repos of a project
in a subdirectory - like "git/" - of course this is completely optional. For a project called "ilight" then:

   cd ~greg/Development
   mkdir ilight             # make a place to do work on a project named ilight
   cd ilight                # here you can keep stuff to do with development not intended for git
   mkdir git                # make a subdir for git repos
   cd git                   # in here make the git clone. 
 

Get a git repo directly from our AFS colletion of bare repos. A repo is a collection of files, and the git 
database wheich keeps track of them. You work on your local 'clone' of a repo. 

   If on an AFS host (like rhel6-64, lcls-dev3, or some other SLAC Public Host): 

      git clone /afs/slac.stanford.edu/g/cd/swe/git/repos/slac/ilight/ilight.git 
      cd ilight

   If on production, you have to ssh through mcclogin. The common login accounts have git set up for that. See:

      $ printenv | grep GIT 
      GIT_SSH=/home/physics/bin/git-ssh
      GIT_SITE_TOP=ssh://@mcclogin/afs/slac/g/cd/swe/git/repos 
      GIT_AUTHOR_NAME=...

      Examples to clone on production hosts then:
      git clone $GIT_TOP/slac/ilight/ilight.git                                    # Using GIT_TOP env var
      git clone ssh:///afs/slac/g/cd/swe/git/repos/slac/matlab/matlabscanner.git   # Repo URL spelt out

     You'll be asked for your SLAC Unix password, after you've given it, and have your files, cd down)
     [physics@lcls-srv01 ~/.../Development/pvnamesfor2020/git]$ cd matlabscanner/

   If on your own laptop (without AFS mounted) and cloning from outside the SLAC network (like at home), use a
   SLAC Public machine in the repo pathname:

      git clone greg@rhel6-64.slac.stanford.edu:/afs/slac/g/cd/swe/git/repos/sites/www.slac.stanford.edu/grp/ad/docs/model/matlab.git
      cd matlab


Git Commands Summary
====================

Make sure you're on the branch you think you're on.

   git branch                       Should say eg "* master" - * indicating you're on the master branch

Check what branches exist for this repository; the -a option lists ALL branches, BOTH locally
pulled/checkedout and those only on the remote repo (shown in red) [NOTE: I have seen a exception to
this, in easyPVAJAva -a does not list all the remotes that -r does!!]

   git branch -a
   
Set the branch you want to work on, e.g. if not master. This also works to bring in remote branches. 

   git checkout release/4.0         (Pay attention to the advice "git checkout" will give you
                                     about the status of your checkout, and issue the "git pull" if necessary)

Check where the origin of a repo is. Eg, in the SLAC reality, a local repo may have origin
in github (possibly in github.com/slaclab) or in gitLab code.stanford.edu, or simply be
rooted at /afs/slac.stanford.edu/g/cd/swe/git/repos/. The golden master should be in the latter,
with optionally a repo upstream of that in either of the former. 

   git remote -v           
   git remote show origin    # gives a bit more, including status of branches

Has anyone changed the remote master repo requiring me to pull or merge?

   git fetch origin; git log HEAD..origin/master

   If other people have changed the remote repo, then (since you just did a git fetch) you can merge in
   their changes with:

      git merge origin/master    

      	            If you had changed your local repo, and there had been remote changes, git will
                    have to do a "real" merge - as opposed to simply applying the remote changes to your
		    local checkout. In that case you will be asked to make a comment - vi exit is ESC wq remember.
		    If you have NOT chaged your local repo, and there had been remote changes, git will
		    instead do a "fast-forward" quasi-merge - just applying the remote changes on top of
		    your outdated checkout.

      git merge --ff-only

                    Ask git to check whether your repo has in fact been changed, and if not simply bring
		    in remote changes. If your local repo has been changed, this command will have no
		    effect and instead you'll have to do the real merge above. So this is a kind of "safe
		    merge for the common case".
                    
   [Not sure how this below is different - it seems to actually fetch and update local repo]
   git checkout master
   git remote -v update

Pull in changes other people made (to the remote version of, by default, your currently set branch)

   git pull [--rebase]              Add --rebase if you want their commit history included in your history
                                    A git pull == git fetch ; git merge
   git pull [--rebase] origin master    If pulling changes from origin master into another branch 
                                        say nttableclarifylabels

See a summary of the changes I've made (on the presently set branch)

   git status                       What files have I changed since my last commit, or commits waiting for push.
                                    Also shows whether the remote repo has been added to since my last push.
                                    'Changes not staged for commit' = You have yet to git add a change. Use git add.
                                    
   git log                          What has anyone committed, ever.

   git log origin/master..HEAD      What commits have you made ahead of orgin/master.
                                    Ie, in the nominal case, what would be pushed were you to simply git push.

   git log --oneline --decorate -n 10 --all
                                    One line summaries of last 10 commits.

  
Diff of files I changed and so are waiting to be added and committed to my staging area (see below):

   git diff             Prints diffs of all files in repo, between what 
                        has been edited, and what has been git added 
   git diff README      Diff a specific file

Diff of a file to that file in a commit in the repo:

   Find the SHA of the commit that has the file version to which you want compare,
      eg: git log --oneline --decorate -n 10 --all  - then with a commit SHA ...
   git diff 21d432d control.html    Prints diff of control.html to its version in commit with SHA 21d432d
   
What would be pushed were I to do the git push:

   git log origin/master..master          What commits have I made (on master) since I last pushed w.r.t. o/m
   git log origin/master..nttableclarifylabels     (what commits have I made on nttableclarifylabels w.r.t. o/m)

   git diff --stat --cached origin/master          (the files to be pushed to origin/master)
   git diff  origin/master                         (the code differences)
   git diff --stat --cached nttableclarifylabels   (the files to be pushed to nttableclarifylabels)
   git diff  nttableclarifylabels                  (the code differences)
   
Which local branch contains a given commit:
   git branch --contains 2d6a0f6                   Uses trick of giving only first 7 chars of commit-id.
   
Which *remote* branches contain given commit (by hashtag) - use to verify whether a commit has been pushed
   git branch -r --contains cbd06e2325d4bd37950ace4f796e724ba7fe63cc


Move files to new directory. Moving can be done 2 ways:

   Using git mv:
   
     mkdir documentation
     git mv RELEASE_NOTES.md documentation      [note I didn't have to git add the directory first]
   
   Using simply unix mv, git add, git rm:
   
     mkdir documentation
     mv RELEASE_NOTES.md documentation
     git add documentation/RELEASE_NOTES.md     Could also have simply "git add documentation" since its recursive.
     git rm RELEASE_NOTES.md
   
Stage. Add changes you've made to the staging area. Note git 'add' command is not at all equivalent to cvs add,
       because you have to git add it *each* time you change a file, not just when you add it to the repo the
       first time. You're adding it to the list of files in the stage "area", which will then be committed
       as one commit.

   git add <file1> [<file2>] ...    Add named files to stage
   git add -u                       Add files modified or deleted' doesn't add new files 
   git add .                        Add all files from root of project incl new files, but not those deleted
   git add -A                       Add all changes, ie git add .; git add -u
   git add <directoryname>          Add all files in directory. No "/" in dirname. 
                                    Can be a pathname. Can be a new directory.
   
   [Can also be done with SmartGit "Stage" button]

   
Commit your staged changes (see git add above) to your *local* branch. Note how git commit is significantly 
different to cvs commit, where the commit is typially to a remote. git push (below) is more like cvs commit.

   git commit -m "comment" 

    Note: 'git add' and 'git commit' can be done in one command with 'git commit -am "comment"'
    Also Emacs Tools->Version Control->Check In/Out does git commit -a

Release your changes to your "upstream repository" - probably one in /afs/slac/g/cd/swe/git/repos/.

   git push         % Push to same repo and same branch as you checked out
                    % On for you'll be asked for your password  

   git push origin my-new-feature-branch     ( push to my-new-feature-branch on the same repo as cloned)
   eg
   git push origin nttableclarifylabels

   NOTE: The first time you use git push, and haven't yet set up your own preferences (~/.gitconfig), you 
   may well get a long warning message beginning "warning: push.default is unset; ...". The simplest
   thing to do is just (once) execute "git config --global push.default simple"
 
   [See also SmartGit "Push" button] 
   

Pull request

   "Pull request" is git speak for asking the owner of a repo from which you have cloned a
   repo, to "pull" changes from your repo, and decide whether they want to ingrate your
   changes into their repo. So it's the basic way of collaboratively developing software,
   where one person owns the golden master repo, and integrates suggestd changes into it.

   So, assuming you forked from a repo under eg https://github.com/epics-base - 
   Check status, and if fine, push to remote branch to origin  

     git fetch origin
     git log HEAD..origin/master
     git push origin nttableclarifylabels

   Having pushed to nttableclarifylabels, can then make pull prequest on git-hub.

   NOTE: There is no specific need to make your own fork in order to create a pull
   request.  One can do just fine using a branch from within epics-base/<repo> (as long as
   you have privilege).  But the EPICS developers do want contributions to be sourced from
   personal (forked) repos - esp Ralph. So, if you do that, instructions at [11].
   
Undoing work

   You edited a file, but are now unsure of your change. You have not yet done git add:

      git show HEAD:<filename>  # Prints (to stdout) the version of filename at the head. You could
                                # replace HEAD with another commit SHA. You can save a copy with redirect,
				# e.g. git show 36bd566:gitcheatsheet.txt > oldgitcheatsheet.txt 

   You edited a file, but want to undo. You have not yet done git add:

      git checkout <filename>

   You edited file(s) and did git add(s), but you want to go back:

      git reset   ( see in particular example at [4])

   You edited, git added, and committed, but you want to revert that commit

      git log [--one-line]       # Find SHA of commit you might want to revert to
      git checkout <SHA>         # Set working repositiory to candidate last good commit
                                 # Look at files and verify files are in in fact good state. If so...
				 # revert the bad commit. 
      git revert <UNWANTED SHA>  # Revert bad commit SHA. This will undo the changes to the files of 
                                 # the named commit. It changes the files back, not just forget that you committed. 
      git status                 # Check the status of HEAD. If it's detatched, eg "HEAD detached at  
                                 # e56f45" then...
      git checkout <current branch> # eg master
      
   You edited, git added, committed and pushed. But you want to go back

      git log                   # See recent (local) commits. Eg See that ccdec1b might be last good commit 
      git checkout ccdec1b      # Checkout commit ccdec1b.
                                # Look around files of ccdec1b, Eg see if ccdec1b has the good
                                # version of file you want. If so othen conclude that ccdec1b is the 
                                # last good version... 
      git checkout master       # Begin process of putting good commit at head of branch. Eg ccdec1b
                                # onto onto master, by setting checkout back to master (or another
				# branch, if doing development on branches)
      git cherry-pick ccdec1b   # Append ccdec1b to working HEAD of master (or whatever branch you set
                                # in line above)
      git status                # get status w.r.t. origin/master, we're ahead by a commit (ccdec1b)
      git push                  # push to origin master
   
   Fully clean a working copy clone directory

      git reset --hard; git clean -f -d; 

      then bring it up to date with git pull.
      
   Recover just an an old revision of a file, without checking out to its commit

      First, git log, to find the commit id (SHA1) of the file - the long hex code. Then do git show and redirect

      git log
      git show 60f1ae1:mainPage/index.html > t.tmp

   Completely remove a file from git's tracking (eg .DS_store files)

      git rm --cached <filename>
      git rm --cached mainPage/talks/.DS_Store

      You must git commit and push before the remote repo forgets the file. Then when collaborators next git pull,
      their instance of the file will be deleted. 

      The 'git rm --cached' only removes the file from tracking (and staging). If you want to remove the file 
      from your directory then also use the shell 'rm' filename> to delete the file.


Use Cases
=========

*  Development of web site. An example of using git for source control and to automatically publish
   following git push from development clone, is at:
      https://www.slac.stanford.edu/grp/ad/docs/model/swdev/wwwslaccheatsheet.txt

Troublesome Circumstances
=========================

* A push of branch was out of sync with the remote. To fix just pull the latest changes (and see above):

   git pull --rebase origin master

* Decided after making commits to master, that changes should really be part of a pull request. 
  To fix, make branch (for pull request) and put work previously committed on origin/master onto it.

    git checkout master
    git log origin/master..master
	        (write down commitnums to be moved)
    git branch newbranch
    git checkout newbranch	    
    git cherry-pick <commitnum>   (commitnum from git log origin/master..master above)   
    git branch --contains 2d6a0f6 (verify the branch the commitnum is now in)

* Accidentally git added a file, and want to undo it. Not yet done a commit

    git reset HEAD <filename>
    
    The filename is as output from git status. Eg

    git status
         On branch release/4.0
         Changes to be committed:
            (use "git reset HEAD <file>..." to unstage)
 
            modified:   pom.xml
            modified:   ....
            new file:   src/services/rdbService/plantoconvert.txt    <===

    git reset HEAD src/services/rdbService/plantoconvert.txt

* Added a post-receive hook to a repo *after* having done first git add / commit / push cycle,
  so now have to redo the past commit, and force push on all files to get the whole thing published/released.

    git commit --amend 'Redo Initial import after fixing post-receie hook'
    git push --force origin master


NOTES
-----
git pull == git fetch + git merge
git pull --rebase == git fetch + git rebase
The second re-writes your history, making it look as if you had committed on top of origin's new branch.


7. REFERENCES
=============

Good summary of git workflow and tagging. Best short description found to date 8-sep-15.
[1] http://rogerdudler.github.io/git-guide/

How we do actual releases - based on the Git Workflow:
[2] https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow

Overall reference on triangular workflow
[3] https://github.com/blog/2042-git-2-5-including-multiple-worktrees-and-triangular-workflows

Good short example of git pull / git pull --rebase / git merge / git rebase: 
[4] http://stackoverflow.com/questions/3357122/git-pull-vs-git-fetch-git-rebase

Good graphical descriptions of rebasing (merging branches)
[5] https://git-scm.com/book/en/v2/Git-Branching-Rebasing

Moving commits between branches (on which recipe for moving change from master to branch for pull-request is based)
[6] http://stackoverflow.com/questions/1628563/move-the-most-recent-commits-to-a-new-branch-with-git

[7]Good guild to pull-request
http://yangsu.github.io/pull-request-tutorial/

[8] Good cheatsheet
https://gist.github.com/hofmannsven/6814451

[9] RELEASE_VERSIONS, https://github.com/epics-base/pvDataWWW/blob/master/scripts/RELEASE_VERSIONS
    
[10] APS f-2-f meeting https://docs.google.com/document/d/1fF6G1dPW7nx5KqgvMnoAR0S9tm-jwjFJcGZ6Zoj6-rE/pub

[11] Forking a repository (in preparation for changing someone elses repo and making a "pull request")
https://help.github.com/articles/fork-a-repo/

[12] Release Tracker, https://docs.google.com/spreadsheets/d/16vIVdopCigfDA7VfepEg4aaKGPqp0n8uIydGIRJpbpg/edit#gid=0