Creating Countinous Integration system with GitLab and Unity

After spending days and nights working in the sweat of our brows and making tons of application builds, we finally decided to take a step forward and make some part of our job automatic.   But...  why ? The main reason of making our Continuous Integration system was shortening amount of time spent on building applications. In our last project it lasted even above half an hour, so we decided to make our life easier. We had a list of features we wanted to achieve:

• builds available online to download
• working on multiple unity versions
• building for iOS, Windows and Android on our Windows device
• builds available on our local server

( Our continuous integration system in few simple steps)   About GitLab CI We chose GitLab CI solution because we had already possessed repositories there. How to configure GitLab and a Runner we described in a separate article:   When we set up our GitLab Runner we had to configure GitLab CI itself by .gitlab-ci.yml Our system works in two stages : build and deploy. Job named “build” is a template for other jobs. It contains a powershell script which run the unity in batchmode and execute script we wrote in C#. To do such a thing we need Unity command lines :  

C:\’Program Files’\Unity\Editor\Unity.exe	call unity executable
-batchmode	execute in batchmode to prevent the editor from opening
-nographics	needed on windows to really make sure there is no GUI
-executeMethod BuildScript.Build	calls the Method “Build()” in the Unity BuildScript
-projectPath %CI_PROJECT_DIR%	sets the unity project path to default GitLab directory
-quit	quit Unity after execution of the build

+ other variables you want to use in unity e.g. -customProjectName %CI_PROJECT_NAME%   GitLab can hands over some variables such as project name, pipeline id, project direction or build platform target. We use them in our unity internal script to build apps for platforms we have chosen [code] .build: &build stage: build variables: tags: - Unity script: - C:\"Program Files"\Unity\Hub\Editor\%UNITY_VERSION%\Editor\Unity.exe -projectPath %CI_PROJECT_DIR% -logfile D:\Logs\%BUILD_TARGET%.log -customBuildPath %BUILD_PATH% -customProjectName %CI_PROJECT_NAME% -customBuildTarget %BUILD_TARGET% -pipelineId %CI_PIPELINE_ID% -batchmode -nographics -executeMethod BuildScript.Build -quit artifacts: name: "%CI_PROJECT_NAME% %BUILD_TARGET% %CI_PIPELINE_ID%" paths: - ./Builds/%CI_PROJECT_NAME%/%CI_PIPELINE_ID%/%BUILD_TARGET% [/code]   Builds on demand  We have four options: Windows, Android, iOS and All, we type them in the “run pipeline” section with a “Target” variable. Unfortunately there is no option (for now) to make a list of a variables, so we need to type them manually. If  we don’t write anything it automatically run a build for all platforms. This is also a reason why job we named “build” is a template, every platform has their own job and is executed only if we write a correct variable. [code] Windows: >>: *build variables: BUILD_TARGET: StandaloneWindows64 BUILD_PATH : ./Builds only: variables: - $Target == "Windows" - $Target == "windows" - $Target == "All" - $Target == null [/code]   Keep it planned All builds can be made on demand, but also after every commit in master branch and everyday at 2 AM. Beside that we are able to configure it in GitLabCI/Schedules.     We are also setting a unity version as a variable on the top of our file to be easily configurable at the start of the project.   Where we can find our builds? We wanted to save builds directly to our local server, but we found out that it was impossible to achieve. We had to workaround it, which we didn't want to use in the first place - copying from a temporary folder on our disc. We were afraid that it will fill the disc too fast with unnecessary data so we had to also clean the folder after completion. That’s what we are doing in the deploy stage. Builds are also available as the artifacts in GitLab.     As fast as possible  To make the whole process faster we decided to start using unity cache server which helps us to save time at changing between different platforms in unity. When it’s working Unity is downloading converted assets from cache server instead of converting them everytime.   Building iOS apps on Windows One of our goals was to build everything on one machine and didn’t involve our mac. The major problem was iOS building, because we thought it was impossible to build an .ipa file on Wndows machine, but we have discovered a plugin that helped us fix the problem. Of course it wasn’t one click configurable, we had to transfer all the certificates from our mac etc. Never the less it was worth the pain. We are also able to use it by command lines so it is perfect for our purposes.   Try it yourself So this is our story of creating continuous integration system with GitLab and Unity. It was harder than it seems to be, but I’m pretty sure that we will keep improving it. The system makes our workflow much more simpler and faster. Don’t wait, go and make yourself your own one!  

I’ve got the power – how to control remotely your PC using Grails

In this article I will show you how to control remotely your windows computer via the local network using Linux machine and Grails application. First we will turn on a device with Wake-On-LAN (WOL) method and then turn it off using RPC shutdown command call. After this few steps, you won't have to push the power button on your computer any more.   1. Turn on a computer - wake up, darling!!!   To begin with we have to check a few configuration things:

• check if BIOS allows to use Wake-On-LAN on the computer. Go to BIOS power management settings and find the Wake-On-Lan configuration there. If related option does not exist it probably means that BIOS automatically supports WOL - most of new hardware does that. When you find an appropriate option check if it is enabled. Depends on your computer motherboard, it is automatically enabled or you will have to set it manually.
• check your system configuration. Open Windows Device Manager, find your local network device. Right click on a device and select "Properties". Next, select Advanced tab, find a "Wake on Magic Packet" option and check if it is enabled.

    Now it's time to write a piece of code which will send a Wake-On-LAN magic packet to your device. For this purpose I have created project in Grails 3.2.0. Create a new service called "WakeOnLanService".   First method converts device's MAC address from String to byte array: [code] private static byte[] getMacBytes(String macAddress) throws IllegalArgumentException { byte[] bytes String[] hex = macAddress.split("(\\:|\\-)") if(hex.length != 6) { throw new IllegalArgumentException("Invalid MAC address.") } try { bytes = hex.collect { (byte) Integer.parseInt(it, 16) } } catch (NumberFormatException e) { throw new IllegalArgumentException("Invalid hex digit in MAC address.") } return bytes } [/code] This method allows to pass MAC address as an argument in two formats "AA:AA:AA:AA:AA:AA" or "AA-AA-AA-AA-AA-AA". Second method generates a magic packet from received MAC address byte array: [code] private static byte[] generateMagicPacket(byte[] macBytes) { ByteArrayOutputStream byteStream = new ByteArrayOutputStream() 6.times { byteStream.write(0xff)} 16.times { byteStream.write(macBytes) } return byteStream.toByteArray() } [/code] Finally the last method is used for broadcasting magic packet to all devices in your local network: [code] static final int PORT = 9 static final String DEFAULT_BROADCAST_IP = '255.255.255.255' boolean sendWOLPacket(String macAddress, String broadcastIp=DEFAULT_BROADCAST_IP) { if(!macAddress) { return false } byte[] wol= generateMagicPacket(getMacBytes(macAddress)) InetAddress address = InetAddress.getByName(broadcastIp) DatagramPacket packet = new DatagramPacket(wol, wol.length, address, PORT) DatagramSocket socket = new DatagramSocket() socket.send(packet) socket.close() return true } [/code] Now you should be able to start your windows machine remotely. PORT and BROADCAST_IP are the default values and can be changed e.g. when you want to send magic packet through a router.   2. Turn off the computer - just stop talking to me   To turn off the Windows machine we will use a shutdown method executed by remote procedure call (RPC). To do that we execute a samba net command: [code] net rpc shutdown -I {ipAddress} -U {username}%{password} [/code] IpAddress is a remote windows machine IP address, username and password are credentials of a windows user with administrative privileges.   Below are the configuration steps needed:

• Firstly, check if your Linux system has samba packages installed. We will use "net rpc" command to communicate with remote computer. For Ubuntu/Debian Linux distribution it is included in samba-common-bin package. To install samba use console command "sudo apt-get install samba-common-bin".
• Configure your windows machine to disable UAC remote restrictions. Locate the following registry subkey: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System. Check if the LocalAccountTokenFilterPolicy registry entry does exist. If not, create a new entry with DWORD value "1".
• check if firewall has opened port 445 for TCP connection. When not exist then you should add a new role in the firewall.

It's time to write our Grails service for turning off the machine. I have created shutdownService file in Grails project which contains following methods:   [code] void invokeShutdown(String ipAddress, String username, String password) { List<String> command = getShutdownCommand(ipAddress, username, password) StringBuilder outputStream = new StringBuilder() try { ProcessBuilder builder = new ProcessBuilder(command) Process process = builder.start() process.consumeProcessOutput(outputStream, outputStream) process.waitFor() } catch (Exception e) { throw new IOException(e.message) } } List<String> getShutdownCommand(String ipAddress, String username, String password) { return [ 'net', 'rpc', 'shutdown', '-I', "$ipAddress", '-U', "$username%$password" ] } [/code]   That's all!!! Now you should be able to control your computer remotely. You can use it in some kind of centralized control panel and control a bunch of computers from one place. You can create a schedule when computers are working and not.

Populate database in Spring

Once upon a time there was BootStrap class. The class was a very friendly class, therefore it befriended many more classes. With more classes inside, BootStrap grew bigger and bigger, expanding itself at the rate of the entire universe. This is not a fairy tale.   This is not a fairy tale, because this is exactly what happened. But first things first, you may wonder what the BootStrap is. It's a mechanism coming from Grails Framework, which executes code on application startup; I'm using it mostly to populate database. Just put the BootStrap.groovy in grails-app/init folder and add some goodies to the init closure. Having a background in Grails, this is something I missed in Spring, especially because as I mentioned before, the code grew fairly big. I wanted to rewrite the whole BootStrap logic in Java, because its older Groovy version somehow reminded me of poorly written tests you may see here and there: verbose and ugly. It just wasn't a first-class citizen of the production code. [java] @Log4j @AllArgsConstructor @Component public class BootStrap { private final BootStrapService bootStrapService; @EventListener(ApplicationReadyEvent.class) private void init() { try { log.info("BootStrap start"); bootStrapService.boot(); log.info("BootStrap success"); } catch (Exception e) { log.error("BootStrap failed," + e); e.printStackTrace(); throw e; } } } [/java] Surprise, surprise! There's an EventListener annotation that you can put on a method in order to track the ApplicationReadyEvent and run some code on application startup. Job's done, right? Well, not really, you CAN do that, but do you WANT TO do that? I prefer to keep the business logic in a service, therefore I created and injected BootStrapService, that just leaves logging and error handling here and Lombok's annotations make the whole thing even neater. [java] public abstract class BootStrapService { @Autowired protected BootStrapEntryService entryService; @Autowired protected MovieService movieService; @Transactional public void boot() { writeDefaults(); } protected void writeDefaults() { entryService.createIfNotExists(BootStrapLabel.CREATE_MOVIE, this::createMovie); } private void createMovie() { movieService.create("Movie"); } } [/java] I made the boot method @Transactional, because it's our starting point to populate database, later in the project you might want to add here some data migration as well. The REAL rocket science begins in writeDefaults method! The example createMovie method reference is passed as a parameter, with double colon as a syntactic sugar, to the createIfNotExists method of the injected BootStrapEntryService. The other parameter is a simple BootStrapLabel enum, with a value used as a description for a given operation. I prefer to add a verb as a prefix, just not to be confused later, when a possibility of other operations comes up. [java] @Log4j @AllArgsConstructor @Transactional @Service public class BootStrapEntryService { private final BootStrapEntryRepository bootStrapEntryRepository; public void createIfNotExists(BootStrapLabel label, Runnable runnable) { String entryStatus = "already in db"; boolean entryExists = existsByLabel(label); if(!entryExists) { runnable.run(); create(label); entryStatus = "creating"; } log(label, entryStatus); } public boolean existsByLabel(BootStrapLabel label) { return bootStrapEntryRepository.existsByLabel(label); } public BootStrapEntry create(BootStrapLabel label) { BootStrapEntry bootStrapEntry = new BootStrapEntry(); bootStrapEntry.setLabel(label); return bootStrapEntryRepository.save(bootStrapEntry); } private void log(BootStrapLabel label, String entryStatus) { String entryMessage = "processing " + label + " -> " + entryStatus; log.info(entryMessage); } } [/java] Finally, createIfNotExists method is the place to call the actual methods to populate database, however, in a generic way. Method passed as a reference may be called, however, we don't want to write the data that was written to the database before, at least considering a not in-memory database, so we check if an entry for a given label already exists. We have to create an entity, a pretty simple entity, in this case the BootStrapEntry, with just a label field to keep the labels in database. existsByLabel and create are simple generic methods responsible for basic database operations on the labels. [java] @Profile("development") @Service public class DevelopmentBootStrapService extends BootStrapService { @Autowired private BookService bookService; @Override protected void writeDefaults() { super.writeDefaults(); entryService.createIfNotExists(BootStrapLabel.CREATE_BOOK, this::createBook); } private void createBook() { bookService.create("Book"); } } [/java] Now we're getting somewhere! If you wondered why I made the BootStrapService abstract, now is the answer. I wanted to make it possible to run some code only in a given environment, like development. The Profile annotation with environment name as a parameter comes in hand. Overriding the writeDefaults method provides a way to initialize some data only in development. [java] @Profile("production") @Service public class ProductionBootStrapService extends BootStrapService { } [/java] If there is the development, there may be the production as well. In the given example, I just wanted to run the default data from the parent class, without filling the environment with some random data.   That's all, my little guide to populate database in Spring. Hopefully, it wasn't THAT bad and even if it was, feel free to leave the feedback nonetheless!

How GitLab helps us move fast at itSilesia

Continuous Integration and Continuous Delivery have taken the world by storm. The nature of our business forces development teams to move quickly and be as efficient as possible, both in regards to standard software development, but also it’s delivery and quality assurance. In order to achieve these goals, we need tools that will enable us to reach them in a simple way. Thankfully, today we have access to a lot of CI/CD solutions, both free and paid, all very different, with different features and different goals in mind. In essence they all do the same thing - allow us to work smarter, not harder.   Our previous development infrastructure consisted of three parts. For issue tracking and Scrum process we used Redmine with Scrum Board plugin. It worked fine and served us great for years, but from today’s perspective it's UI is seriously outdated and generally hard to use. We probably could give it a try to upgrade it, but sadly it is so old that we really do not want to try it. For hosting Git repositories we used Gitolite Redmine integration. It also worked fine, but it's functionality is nonexistent compared to modern solutions, it lacks really basic functionalities like Pull Requests or commenting on commits. For continuous integration we were (and somewhat still are) using Jenkins. Now, don't get me wrong - Jenkins is great and very powerful, requires minimal setup, can be scaled and we achieved some great success with it, especially with use of Groovy Pipelines. But here is the question, if there is a tool (GitLab, if you didn't read the title) that could integrate all of the above, even for some loss of customizability and potential vendor lock-in, is it worth it? Well, let's check it out!   1. Setup How do we start then? The first decision to make is whether we want to use GitLab.com hosted service, or create our private GitLab instance on company infrastructure. This decision was simple to make - we don't want to store the most valuable company asset (the source code) on public infrastructure, and given the recent problems with GitLab accessibility - this is just a better option. There are multiple ways of installing GitLab, but the easiest way for us was using prebuilt "batteries included" Docker image, which contains all services in one big bundle. This way of packaging multiple applications into one monolithic Docker image is a bit controversial in the community (the best practice is to join individual processes running in separate containers using Docker Compose, for example), but in my opinion it works great for such big and complex software packages, because all the internal complexity is completely hidden away. As far as installation is concerned, we already have a few Docker hosts, so running an additional container somewhere isn’t a big deal.   Documentation specifies that the following Docker invocation is a good start: [code] sudo docker run --detach \ --hostname gitlab.example.com \ --publish 443:443 --publish 80:80 --publish 22:22 \ --name gitlab \ --restart always \ --volume /srv/gitlab/config:/etc/gitlab \ --volume /srv/gitlab/logs:/var/log/gitlab \ --volume /srv/gitlab/data:/var/opt/gitlab \ gitlab/gitlab-ce:latest [/code] As with everything shipped in a form of a Docker image, we could just blindly copy and paste it into a privileged console... but let's slow down. Do we really understand what these parameters mean? First of all, on most Linux systems port 22 is already used by SSH server, so another binding on port 22 will surely conflict. SSH port is used by GitLab to run it's internal SSH Git server. Since we are perfectly happy with using only HTTPS as before, we can remove this binding altogether. Also, in our case (and in most cases on typical production systems) HTTP(S) ports (80 and 443) are already taken by some Apache or Nginx web server running natively. Since we wanted to use our external Apache proxy (which was also doing SSL termination) on the company edge router, we had to change HTTP port binding to some other random-ish value. Also, we can remove the hostname, it seems not to affect anything.   After a few minutes of some heavy internal provisioning and initialization, you can visit the main page. It will ask for an administrator password first, then it will allow to log in. I have to admit that I was very pleasantly surprised by how easy this setup process was.     I will not get deep into typical administrative stuff. As a first thing you probably want to create users, assign them to groups, create repositories and hand out permissions. It's not really interesting to be honest.   2. Projects The next topic is migrating projects. Since Git is a decentralized system, it doesn't really matter how many repository servers are used for a single project. However, even though we could work this way (and technically we are), we generally don't think in a decentralized way - typically there is just one central repository. This becomes a challenge, when you want to migrate existing projects when developers are working on them at the same time. The first step is fairly easy - open a console (you are using Git from console, right?), create a new Git remote, and then push all branches to newly created GitLab repository. In the following examples I'll use a very simple Spring Boot application (you can find it here). [code] cd ~/luckynumber git remote add gitlab \ https://gitlab.gliwice.itsilesia.com/agrzybowski/luckynumber.git git push -u gitlab --all git push -u gitlab --tags [/code] After some uploading, you will end up with two identical repositories.   And...     The issue is that now we have 2 remotes, which means that if you previously had branch develop (and it's remote tracking branch counterpart origin/develop), now there is a new tracking branch gitlab/develop. And that's only on your local repository - other team members can't know (by definition of decentralized model) that there is another remote somewhere. There are two ways of dealing with this. The easiest way is to go around the office and yell "Guys, please copy your files, delete the project and reclone from scratch". And this might work, but expect a lot of hasty copy-pasting and useless bulk commits after this procedure. There is of course a better way. Git has an option to simply change the remote URL. Send your coworkers a Slack message to run this: [code] cd ~/luckynumber git remote set-url \ https://gitlab.gliwice.itsilesia.com/agrzybowski/luckynumber.git [/code] Be careful, though - all remote branch references (origin/xxx) will immediately get out of sync, so push their local counterparts as soon as possible. It’s a good idea to treat remote branches as volatile and not get too emotionally attached to them, because they can be overwritten at any time (remember, git rebase is your best friend, if used correctly). If they are important, don’t use them - just creating a local branch pointing to some commit will make it persistent forever.   3. The interesting part Now let's focus on CI/CD part. Glancing over the docs, the first thing that pops up is that GitLab CI is very tightly integrated to the source code repository. All configuration (in form of YAML file) is stored directly in the repository, jobs are based on branches and triggered on pushes, pull requests have little red/green checkmarks with build statuses and so on. Is this a good idea altogether? In my opinion it is, but only for the CI part, like running tests or collecting code quality metrics. Deployment (especially on production environments) should be decoupled from any version control system the application happens to use. But since we can get away with this in our internal projects (we have relatively little risk and hopefully responsible developers), we decided to go 100% GitLab and do our production deployments also there.   GitLab CI configuration is stored in .gitlab-ci.yml file. It has a specific structure, which we will not cover in full here. The documentation is very comprehensive, and you can also find some examples online. We will be building an almost barebones Spring Boot application with a few unit tests. [code] image: openjdk:8-jdk stages: - test - package - deploy test: stage: test script: - ./gradlew test package: stage: package only: - master - develop script: - ./gradlew bootRepackage artifacts: paths: - build/libs/luckynumber-0.0.1-SNAPSHOT.jar deploy-test: stage: deploy only: - develop script: - ./deploy-test.sh deploy-prod: stage: deploy only: - master script: - ./deploy-prod.sh [/code] There is a lot of stuff going on here, let's break it down. First of all, the recommended way of building/testing/packaging your application is by using ephemeral Docker containers for creating temporary and 100% reproducible build environments. We went through a lot of pain on traditional Jenkins because of conflicting JVM versions, badly installed Grails distributions or even globally installed NPM packages (which "no one remembers installing"...). Using Docker containers removes this problem completely - every time a build is triggered there is a guaranteed fresh environment available. Here we specify OpenJDK public image, but you can use any, even your own. After setting up the image name, we define build stages. We use very typical steps - build/package/deploy, but they are of course arbitrary. For each stage we can define:

• branch constraints - for example, deploy to production only from master,
• artifacts - they are preserved across stages and stored internally in GitLab for later download,
• a Bash script - defining what to actually do.

  I have to admit that I really dislike writing long shell scripts as a list of YAML elements (even though online examples seem to encourage this) - some characters like & have special meaning in YAML, which means you have to use strange escape sequences, and UNIX gods help you if you have to create some nested command with sed and regexes for example. For this reason, the best way to do it is to write separate scripts and commit them to the repository, then invoke them from YAML script. And that’s exactly what we do here.   After committing .gitlab-ci.yml file to repository you can head to the tab CI/CD and look at a newly triggered job. It will be in a paused state... because we didn't tell GitLab yet how to run our builds! For this, we will have to add a so-called GitLab Runner. What is a Runner? In essence, it is a machine which will execute our jobs. It can be a physical server, a virtual machine, or even a throwaway AWS EC2 instance running Docker Machine on autoscaling cloud infrastructure! Okay, getting back on the ground, since our needs are rather modest, we decided to use a small KVM virtual machine, running on the same server GitLab itself is running on. It's explicitly discouraged by the documentation for management and security reasons, but we have 24 GiB of RAM and a lot of disk space there, so there shouldn't be any capacity problems, and top performance isn't really needed. We will be using a Docker executor, which is just a small service that needs access to Docker daemon on the underlying host. [code] sudo wget -O /usr/local/bin/gitlab-runner \ https://gitlab-runner-....../gitlab-runner-linux-amd64 sudo chmod +x /usr/local/bin/gitlab-runner sudo useradd \ --comment 'GitLab Runner' \ --create-home gitlab-runner \ --shell /bin/bash sudo gitlab-runner install \ --user=gitlab-runner \ --working-directory=/home/gitlab-runner sudo gitlab-runner start [/code] And as a last step, we need to register the runner, so it becomes visible in GitLab and it can be used to schedule jobs. Of course, you can add more than one runner - GitLab will automatically load-balance jobs across them. You can also do some fancy stuff, like locking a runner for specific project or attaching tags for easier administration. Here we just answer the questions one by one. We can obtain GitLab-CI token for a shared runner from administration page (Admin area > Runners) and paste it here, it needs to be done only once. We can safely enable this runner to run multiple projects and run all jobs. [code] $ sudo gitlab-runner register Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com ) https://gitlab.gliwice.itsilesia.com Please enter the gitlab-ci token for this runner ... Please enter the gitlab-ci description for this runner [gitlab] its-runner Please enter the gitlab-ci tags for this runner (comma separated): Whether to run untagged jobs [true/false]: [false]: true Whether to lock Runner to current project [true/false]: [true]: false Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell: docker Please enter the Docker image (eg. ruby:2.1): debian:sid Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! [/code] Right after applying those settings our newly created pipeline should kick off automatically. Opening CI/CD tab, we can see the individual pipeline stages, and show the console output.   And what about deployment? Surely we need to get access to some production servers via SSH or in some other way? That's the problem we discussed earlier. Should GitLab have access to external servers at all? That's a tough question, and there is no easy answer for all use cases. We decided that the gained simplicity and full CI/CD integration is worth the extra risk. But in any case we absolutely shouldn't store any credentials or passwords inside repository, ever. GitLab has a simple Secrets feature, which allows us to define special variables, that at the time of job execution will get injected into environment. We defined a single SSH_PRIVATE_KEY variable, containing a private key, that will be subsequently used to access our test server. But there is a catch! All jobs always run in containers, and we need to make sure that our container has all the tools necessary to perform a deployment, even basic ones like ssh-agent or bzip2. We can use a package manager to quickly install what’s missing in the image, or simply use a bigger image (like ubuntu) with all tools preinstalled.   For our SSH deployments, we created simple script (idea was taken from the official docs) that configures ssh-agent with a private key required to access the server. This simple script starts ssh-agent, disables SSH Host Key Checking and adds a private key to the in-memory store. Disabling host checking is sadly necessary, otherwise we would have to manually approve connections every time (not once - remember the part about fresh containers?), and we can't do that - there is no keyboard/terminal input when job runs. [code] #! /bin/bash set -e echo "Configuring ssh access..." which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y ) eval $(ssh-agent -s) ssh-add <(echo "$SSH_PRIVATE_KEY") mkdir -p ~/.ssh echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config echo "SSH configured successfully, added ssh keys:" ssh-add -l [/code] And we are basically done! From this moment, every push to the repository will trigger a pipeline run. Tests will be executed for all branches (including feature ones), and an appropriate package+deploy procedure will be ran for develop and master branches.   There is still one last thing missing: the actual deploy script. This is of course dependent on the type of application and target environment. For frontend applications, we might build necessary assets using NPM and Webpack, then upload them through (S)FTP to a specific folder for Apache to serve. For Java apps deployed in Tomcat, we might build the .war file and put it into Tomcat directory, where the change will be detected and the app restarted. For Docker Swarm environments, we can build all necessary images, push them to private registry and trigger docker service update or docker stack deploy on the remote cluster manager node.   That’s everything we have for now. We’ve been using this setup for one of our legacy projects for a few months and it works very well. Our development team is very happy to have one, integrated experience and very visible progress. Stay tuned for updates!   But wait, there’s more!   We wanted to host GitLab on company network and expose it to the world through external HTTPS proxy. At first, it seemed that setting instance URL in administration panel to its HTTP equivalent is enough, because it worked for cloning and browsing. Unfortunately, there is a nasty hidden problem in doing only this. You see, when GitLab is accessed through external HTTPS proxy, it can't be aware of this fact, and as a result all links inside GitLab have http:// and not https:// prefix. Not a big deal - developers can change the URL once when cloning and then forget about it, yes? Well, that's technically true, however if you try to kick off any CI build you will find out that it mysteriously fails with empty page without any errors. It turns out that in some very rare cases the frontend part (and also the runner part) of GitLab actually uses the real configured URL of GitLab to retrieve data from the server. We need to tell GitLab it's real external URL - but there is another catch! If you just specify https:// URL per documentation, it actually disables HTTP completely. Argh! The following manual configuration solves this problem. [ruby] external_url 'https://gitlab.gliwice.itsilesia.com' nginx['listen_port'] = 80 nginx['listen_https'] = false nginx['proxy_set_headers'] = { 'X-Forwarded-Proto' => 'https', 'X-Forwarded-SSL' => 'on' } [/ruby]

Star Wars opening crawl based on CSS animations and transformations

10 years ago nobody expected how frontend is going to evolve in the future. Developers from around the world were using CSS for basic styling because then only that was possible with CSS. Complex things, like animations or transformations, were made thanks to plain Javascript or, very popular at this time, jQuery framework.   Currently, with CSS 3, HTML 5 and tons of Javascript frameworks for every possible use, we are in a totally different programming world. CSS has immense ecosystem of styles, which help us create, color, filter, transform or even animate objects on a screen. In this article I want to show a simple way to create animated Star Wars opening crawl using only HTML and CSS. There are a couple of methods to achieve this goal but we will try to choose the one that has the best performance. You may ask: "Why Star Wars opening crawl? There are so many interesting topics to choose from." Yeah, it's true but this task is relatively easy to implement and, what is more important,  we need the most important CSS styles for animations and transformations to achieve it. And I love Star Wars.   1. Setup At first, we have to prepare HTML and basic CSS for displaying opening crawl. The only thing we need in HTML body is this simple piece of code: [html] <p>Here put your opening crawl story</p> [/html] Now we need to apply basic styles to the background of our scene and to the crawl's content. Original Star Wars crawl contains blackish background and yellow text sliding on a screen.  This can be done simply by adding this CSS code: [css] body { background: #111; } p { margin: 0 auto; color: #fcd000; font-size: 30px; font-weight: bold; width: 600px; } [/css] Nothing special - just yellow text on a black background, you probably did it in your primary school. Fortunately, now we are going to use all the magic of CSS to make it ride.   2. Transformations As you can see on the featured photo, our text must be leaned. To make it work we need one simple but very powerful style - transform. It prpvides us a lot of transformation functions but the most important are these, which are connected with translation, rotation and scaling. Currently, on most browsers we can use it in 3 dimensions (x, y, z).   In our case try to imagine that you put our <p> text into XYZ plot. We want to lean this text forward, in mathematical words, rotate around X axis.   And this is also pretty easy to do with CSS - just add this chunk of code to <p>: [css]transform: rotateX(30deg)[/css] If you try to write the code along with me, you can see that this is certainly not what we expected. It looks more like shrinking than rotating with perspective. Why? The keyword here is the perspective. We didn't declare this perspective, so browser did it for us and set perspective to none. As a result, there isn't any visible perspective. How can we declare a perspective? There are two methods which are a little bit different. We can attach perspective to the transformed object or a parent of this object. In that case there is no difference but it's extremely important to use the second approach if you want to transform more than one object. If you attach perspective to the parent, all children will belong to one perspective. And this is how it works in real life! On the other hand, you may find situations where "perspective per object" approach is better - mostly when you don't need realistic behaviour but only nice looking effect.   In our task we will use parent's perspective. To do this we have to put the code below into our styles (in our case these are <body> styles). For the sake of testing try this value: [css]perspective: 50px[/css] I must be honest, for a long time (waaaaay too long) I was writing random pixels' value into this property and checking if it's correct. But inside, I've always wanted to understand what these pixels mean. Nowadays, my beard is thicker, I switched from tea to coffee and finally I understand perspective property, yay! MDN documentation explains, that this value is a distance between user's eyes and z = 0 value on invisible plot. I will try to explain it more clearly. If you make perspective value very small, it looks like you are standing very close to the origin of object. Vice versa - if you provide really big value, you see an object from a really far distance. With this knowledge you can estimate what value you should type here. In our case our block of text is 600 pixels wide. We want to get the effect of seeing the crawl very close, so we should consider value, which is much lower than this 600 pixels. In our test we used 50 pixels but we can see that we "are standing" almost inside the block of text, so we should increase it to at least 150 pixels.     That should create a pretty nice effect of static Star Wars opening crawl. [css] body { background: #111; perspective: 200px; } p { font-size: 30px; color: #fcd000; font-weight: bold; width: 600px; margin: 0 auto; transform: rotateX(30deg); } [/css]   3. Animation We've got the text on a screen, it's leaned as we wanted but we still need to make it move. At this moment we have to decide which way we choose - and this will result in how efficient our code will be. Practically, we have three significant methods to do this - 1. margin-top property, 2. position absolute/fixed with top property or 3. transform property using translate relative to Y axis.   Modifying margin is the worst choice. Seriously, never ever do that! Margin property was never created by browsers developers to use it in complex animations. Okay, in this simple example of one animated block of text, it's possible that you won't see any difference. But try to animate hundreds of objects on a screen with margin and you will see that you should forget about this method. And if you need just a simple animation it's still not a good choice. Why would you use something worse if you have something better? Speaking about something better..   ..top property. Certainly, it is a better option, but still not the best. In the current frontend world there are many popular applications which use top/left properties to animate movement on scrollbars, sliders or animated dropdowns. But these examples are relatively simple tasks for our powerful devices (don't tell me your 2014 smartphone is not powerful, it is powerful enough). It has one advantage over next method - it's more compatible so if you need an effect which works on older browsers maybe it's better.   For complex visualizations or simple games using many animations you should always choose transform over margin or positioning. Browsers have different approaches to render DOM but mostly they use a different way to render elements transformed with transform property. For example, on Chrome, elements are taken into its own layer of GPU (RenderLayer), what causes frames to be drawn quicker. If you are interested in understanding it more precisely, try to make a simple animation with position absolute and check "Performance" section in Chrome Developer Tools. Then do the same with transform property. You will see differences in rendering times and GPU layers used to render it.   Okay, so we all agree that in that case transform is the best choice. We will stay with the transform property because it will be rendered on a separated GPU layer and we will create a crazy fast opening crawl animation. But you may ask: "Where is this animation mechanism?" Transform on its own just sets specific translation, rotation or scale, STATICALLY, nothing more. To make it move we need animation property - next fancy CSS feature. And this is quite tricky and complex feature. With animation property we can define duration, delay, name, iteration count, direction, even timing function of animation! But to simplify, we will leave most of these properties default. Without more ado, let's make it moooove. Add this code into the <p> styles: [css]animation: animateCrawl 20s[/css] We declared that this block will use animation called "animateCrawl" and it will take 20 seconds. But we still didn't define this animation and to do this we need to fill our animation's keyframes. [css] @keyframes animateCrawl { 0% { transform: rotateX(30deg) translateY(400px); } 100% { transform: rotateX(30deg) translateY(-300px); } } [/css] For those who don't understand what does this syntax mean - first of all we need to enter keyframes expressed in percents. Then we define object's styles in these moments. Animation mechanism will automagically interpolate the difference in time what causes the animation effect. In result, our block of text will move according to Y axis, from the bottom to the top.   If you are observant, you may see that I also added rotateX value into transform property, which was added before. At the beginning of my frontend adventure I was asking myself: "Why can't I use only translation here if I want to animate only translation?". Answer is simple - the consequence. CSS treats every style as a pair - property name and value. Transform is just a single property name, so if you want to make a few transformations, you must put them all into this transform style. If you type only translateY transformation, you will simply override previously added rotateX with a default value. Not very comfortable but we must accept what CSS gives us.   If you launched this animation, you may see a small problem - at the beginning the movement is really fast and it slows down after every frame. From physical perspective it is done properly but original Star Wars opening crawl was moving uniformly. To fix it we need to change animation timing function. It gives us a lot of freedom, we can even define our own cubic bezier function! To simplify we will use built-in timing function called "ease-in". Mechanism of "ease-in" is pretty easy to understand - every frame animation is faster than frame before. And this is exactly what we need. Perspective gives an impression of slowing down, "ease-in" function accelerates in time so in result we get steady movement. Of course, if you want to make it very precisely, you should consider using custom cubic bezier function. [css]animation: animateCrawl 20s ease-in;[/css] And it's done. Star Wars opening crawl. Now you can use this knowledge for various CSS animations.   If you want to see working solution, check this codepen: https://codepen.io/Mossar/pen/rGpWqX

Refreshed image

A polish classic says: "sit deeply in an armchair, tighten your belts and start". We do not have a ribbon to cut, so another chapter in company's history will be immortalized by an inaugural entry on a blog.   itSilesia has metamorphosed and returns with a new image. We performed fundamental changes - rebranding met a building renovation and a new blood in our team. We count on that the new website and company materials' design will simplify access to any information you need.