1 # Release of a SoT package on ROS {#tutorial_release_sot_packages}
3 \section Tutorial_introduction Introduction
5 This tutorial will help to realease any new SoT package and to maintain already released package on ROS.
7 Before any release you should test the prerelease of the package to see if there is no errors remaining.
9 \section prerelease Prerelease on the local machine to test for issues
11 \subsection Prerequisite
13 Docker need to be installed on the computer (ref.: https://docs.docker.com/engine/install/ubuntu/ ) :
16 sudo apt-get install docker-ce docker-ce-cli containerd.io
18 Test if the docker engine is correctly installed:
20 sudo docker run hello-world
23 \subsubsection ROS Set up your PC for a prerelease if ROS is already installed
25 Install python3 ros buildfarm in order to generate the script needed for the prerelease :
27 sudo apt-get install python3-ros-buildfarm
29 \subsubsection not_ROS Set up the PC for a prerelease if ROS is not installed yet
31 Setup of apt-get sources to include packages.ros.org. Run the following commands :
33 sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
34 sudo apt-key adv --keyserver hkp://pool.sks-keyservers.net --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
37 \subsection prerelease_ros Creating a Prerelease Command
39 Go on this site : http://prerelease.ros.org/
41 Choose your ros distro
43 Then select the package you want to test in the scrolling menu if your package is already on the buildfarm. If not push the red cross and add a custom repository.
47 After a little time the site search for dependencies to test and then you can push the generate command button.
49 Copy/paste the code on the computer terminal.
51 It should create 5 .sh files inside /tmp/prerelease_job/
53 Launch the prerelease with :
57 If everything went well you will arrive until the 11 job where the test should be done. We are not using those catkin test for the SoT packages. You should have the prerelease finishing with :
64 # BEGIN SECTION: Create collated test stats dir
68 Test results of underlay workspace
70 Invoking: catkin_test_results /tmp/prerelease_job/ws/test_results --all
71 Test results directory "/tmp/prerelease_job/ws/test_results" does not exist
74 Skipping empty overlay workspace
76 If not search what is wrong with the package. It could be :
78 - Missing dependencies
82 \subsection prerelease_error Possible errors during the prerelease
86 Permission denied to access config.json
90 sudo chown "$USER":"$USER" /home/"$USER"/.docker -R
94 permission denied for the creation of .ccmake/tmp
98 sudo chown -R $USER:$USER /home/$USER/.ccache
100 If a new issue is met : https://docs.docker.com/
102 \subsection clean_docker Clean docker
104 After a prerelease there are still generated files left on the computer and it will soon take too much space.
108 docker system prune -a
110 To remove all the unused images, containers, volume and network of docker.
112 \section first_time_release Release a package for the first time
114 \subsection prerequisite_intro Prerequisite
116 Bloom need to be installed (ref.: http://ros-infrastructure.github.io/bloom/ ) :
118 sudo apt-get install python-bloom
120 To allow bloom to generate automatic pull request make sure to have a file ~/.config/bloom well initiated :
123 "github_user": "<github-username>",
124 "oauth_token": "<token-you-created-for-bloom>"
127 Log in to the GitHub account and generate a new token with “public_repo” granted.
128 It is situated inside the GitHub account settings under Developper settings and Personal access token
130 Copy this code inside the bloom file. For example:
133 "github_user": "Rascof",
134 "oauth_token": "295a7188fbfc36f9esdsdsdqssjd21s2d1"
137 If you have a two-factor authentication enabled follow this tutorial (Step 2) : http://wiki.ros.org/bloom/Tutorials/GithubManualAuthorization
139 Make sure that the .gitmodules file is clean without unmet path target. Because it can result in errors during the release.
141 \subsection release_repository Create a github repository for the release
143 For the first release, create a github repository with the name : <package-name>-ros-release
145 Check the Create README.md button when creating it so that the first commit is created.
147 Create a tracks.yaml file inside the repository like this (example at : https://github.com/ipab-slmc/eigenpy_catkin-release/blob/master/tracks.yaml) :
152 - bloom-export-upstream :{vcs_local_uri} :{vcs_type} --tag :{release_tag} --display-uri
153 :{vcs_uri} --name :{name} --output-dir :{archive_dir_path}
154 - git-bloom-import-upstream :{archive_path} :{patches} --release-version :{version}
156 - git-bloom-generate -y rosrelease :{ros_distro} --source upstream -i :{release_inc}
157 - git-bloom-generate -y rosdebian --prefix release/:{ros_distro} :{ros_distro}
158 -i :{release_inc} --os-name ubuntu
159 - git-bloom-generate -y rosdebian --prefix release/:{ros_distro} :{ros_distro}
160 -i :{release_inc} --os-name debian --os-not-required
161 - git-bloom-generate -y rosrpm --prefix release/:{ros_distro} :{ros_distro} -i
164 last_release: v<MAJOR.MINOR.PATCH>
165 last_version: <MAJOR.MINOR.PATCH>
169 release_repo_url: https://github.com/stack-of-tasks/<package-name>-ros-release
170 release_tag: v:{version}
171 ros_distro: <ros_distro>
173 vcs_uri: https://github.com/stack-of-tasks/https://github.com/stack-of-tasks/<package-name>
178 - <ros_distro> to the distro version wanted. To release to several versions just copy past and change the <ros_distro>
180 - <MAJOR.MINOR.PATCH> to the newest version of the released package
184 \subsection release Releasing the package for the first time
186 # Replace <ros_distro> with the ROS distribution, e.g. melodic; noetic
187 # Replace <package-name> with the name used in the ROS distro file
188 bloom-release --rosdistro <ros_distro> --track <ros_distro> <package-name>
192 bloom-release --rosdistro <ros_distro> --track <ros_distro> <package-name> --edit
194 To edit the tracks.yaml file (ref : https://wiki.ros.org/bloom/Tutorials/FirstTimeRelease#Releasing_Your_Packages )
196 There will be this error showing :
198 Specified repository 'dynamic-graph-release' is not in the distribution file located at 'https://raw.githubusercontent.com/ros/rosdistro/master/melodic/distribution.yaml'
199 Could not determine release repository url for repository 'dynamic-graph-release' of distro 'melodic'
200 You can continue the release process by manually specifying the location of the RELEASE repository.
201 To be clear this is the url of the RELEASE repository not the upstream repository.
202 For release repositories on GitHub, you should provide the `https://` url which should end in `.git`.
203 Here is the url for a typical release repository on GitHub: https://github.com/ros-gbp/rviz-release.git
204 ==> Looking for a release of this repository in a different distribution...
205 No reasonable default release repository url could be determined from previous releases.
206 Release repository url [press enter to abort]:
208 This is ok because the package is not yet on the build farm
212 https://github.com/<name-of-account>/<package-name>-ros-release.git
214 Don't forget the .git at the end.
216 Enter the login and password for the release repository to push different branches.
218 Then respond y in order to add information:
220 - For documentation on the package:
222 VCS Type must be one of git, svn, hg, or bzr.
224 VCS url: https://github.com/stack-of-tasks/<package-name>.git
225 VCS version must be a branch, tag, or commit, e.g. master or 0.1.0
228 - For source entry on the package:
230 VCS Type must be one of git, svn, hg, or bzr.
232 VCS url: https://github.com/stack-of-tasks/<package-name>.git
233 VCS version must be a branch, tag, or commit, e.g. master or 0.1.0
236 - For testing and maintenance statue
238 Would you like to turn on pull request testing? [y/N]? n
239 Would you like to add a maintenance status for this repository? [Y/n]? y
240 Please enter a maintenance status.
241 Valid maintenance statuses:
242 - developed: active development is in progress
243 - maintained: no new development, but bug fixes and pull requests are addressed
244 - unmaintained: looking for new maintainer, bug fixes and pull requests will not be addressed
245 - end-of-life: should not be used, will disappear at some point
248 Then at the end it should ask to generate a pull request to add the package to the buildfarm.
252 If there is an error at this stage, make sure to have a ~/.config/bloom well initiated or follow this tutorial if you have two-factor authentication enabled : http://wiki.ros.org/bloom/Tutorials/GithubManualAuthorization
256 ! [remote rejected] bloom-sot-core-0 -> bloom-sot-core-0 (refusing to allow a Personal Access Token to create or update workflow `.github/workflows/build_test.yaml` without `workflow` scope)
257 error: failed to push some refs to 'https://678b1f245c644b91864fd46c9732368c0ab93f0e:x-oauth- basic@github.com/nim65s/rosdistro.git'
258 Failed to open pull request: CalledProcessError - Command 'git push https:// 678b1f245c644b91864fd46c9732368c0ab93f0e:x-oauth-basic@github.com/nim65s/rosdistro.git bloom-sot- core-0' returned non-zero exit status 1
260 Create another token with workflow enabled.
263 \section maintained Maintain packages already on the buildfarm
267 # Replace <ros_distro> with the ROS distribution, e.g. melodic; noetic
268 # Replace <package-name> with the name used in the ROS distro file
269 bloom-release <package-name> --rosdistro <ros_distro>
271 Enter the login and password two times for the release repository to push different branches.
273 At the end it should generate a pull request on GitHub to update the version on the GitHub repository of ROS : https://github.com/ros/rosdistro
275 If not make sure to have a ~/.config/bloom well initiated or follow this tutorial if you have two-factor authentication enabled : http://wiki.ros.org/bloom/Tutorials/GithubManualAuthorization
277 \section patch ROS Patch
279 This section shows how to manage patches. If there is an installation rule specific to ROS you can use patches. Patches are a rule to modify the source code only when installed with ROS.
280 For more insight : http://wiki.ros.org/bloom/Tutorials/ReleaseThirdParty from section 2.4 to 2.6
282 \subsection patch1 Create a patch
284 Clone the ros release repository on the computer
286 git clone https://github.com/<name-of-account>/<package-name>-ros-release
287 cd <package-name>-ros-release
289 Enter the branch of the ros release repository where the source code need to be changed
291 git checkout release/<ros_distro>/<package-name>
293 Make the modifications
296 git commit -m "<Comments>"
297 git-bloom-patch export
299 git-bloom-release <ros_distro>
301 Finally, procceed with a realease on ROS.
303 \subsection patch2 Remove patches
305 There is a way to remove all patches (I did not find a way to remove only one patch)
307 Clone the ros release repository on the computer
309 git clone https://github.com/<name-of-account>/<package-name>-ros-release
310 cd <package-name>-ros-release
312 Enter the branch of the ros release repository where the source code need to be changed
314 git checkout release/<ros_distro>/<package-name>
318 git-bloom-patch remove
320 git-bloom-release <ros_distro>
322 Finally, procceed with a realease on ROS.
324 \subsection patch3 Porting patches between ros version
326 Port patches from one distro to another
328 git checkout patches/release/<new_ros_distro>/<package-name>
329 git ls-tree --name-only -r patches/release/<former_ros_distro>/<package-name> | grep '\.patch' | xargs -I {} sh -c 'git show patches/release/<former_ros_distro>/<package-name>:"$1" > "$1"' -- {}
331 git commit -m "Importing patches from <former_ros_distro> to <new_ros_distro>"
332 git checkout release/<new_ros_distro>/<package-name>
333 git-bloom-patch import
337 Finally, procceed with a realease on ROS
339 \section check Check if the build succeded on the build farm
341 When the release is done and the pull request is accepted by ROS developpers the package will automatically be build on the ROS build farm with Jenkins.
343 The maintainer of the GitHub repository specified inside the package.xml will receive an e-mail with the success or failure of the build.
345 There is a need to check if all jobs succeded on the site where the ROS build farm is held : http://build.ros.org/
347 Search for the name of the package and check every job.
349 The first letter of the jobs is the ROS distro version :
354 \section TestingRepository Testing Repository
356 When the package is finally released it is not yet on the official build farm. It is in a soaking area where you can test your packages before it is released on the public ROS build farm. For more insight : http://wiki.ros.org/action/show/TestingRepository?action=show&redirect=ShadowRepository
358 To change to ros testing :
360 Change ros into ros-testing inside this file /etc/apt/sources.list.d/ros-latest.list
364 deb http://packages.ros.org/ros-testing/ubuntu bionic main
369 sudo apt-get install ros-<ros_distro>-<package-name>
371 To go back to the official ROS build farm change back to ros on /etc/apt/sources.list.d/ros-latest.list
376 sudo apt-get dist-upgrade