How to connect to the Little Big Data cluster Web services over ssh

To view your Yarn jobs log files in the browser you will need to connect to some web services that are only accessible over ssh.

Connect with ssh dynamic port forwarding and Socks5 proxy in the browser

To connect this way you need to:

1. build a tunnel over ssh using:
   • command-line ssh
   • putty
2. configure a proxy in the browser

1a. Build tunnel for command-line ssh users

Add this to your ~/.ssh/config file:

Host lbd
    HostName lbd.zserv.tuwien.ac.at
    ForwardX11 yes
    User <yourLoginName>
    DynamicForward 9918

Connect to the server by typing:

$ ssh lbd

As long as the terminal window running ssh is open, you have an active tunnel.

1b. Build tunnel for putty users

• Session -> enter lbd.zserv.tuwien.ac.at as Host Name, 22 as port
• Session -> Connection type: ssh
• Connection -> data -> enter login name as User name
• Connection -> ssh -> X11 -> select "Enable X11 forwarding"
• Connection -> ssh -> tunnel -> Add new forwarded port/source port: 9918 and click "add"
• Connection -> ssh -> tunnel -> Select "Dynamic"

2. Setup Socks5 proxy in browser or in system settings

Create a file called proxy.pac containing the following Javascript code (note: the port number should be the same as in the config script):

function FindProxyForURL(url, host) {
   if (shExpMatch(host,"*.local")) {
     return "SOCKS5 localhost:9918; ";
   } 
   // Alle anderen Anfragen
   return "DIRECT";
}

This file is also available at http://lbd.zserv.tuwien.ac.at/lbd.pac . In case you are concerned about security using proxy files, you might prefer the local file.

In your Firefox browser open:

Preferences -> Network Settings -> Settings

Select "Automatic proxy configuration URL" and input either

• file:///path/proxy.pac in the next input field (change this to the path of your proxy path file -- notice there are three slashes after file: because one belongs to the path), or
• http://lbd.zserv.tuwien.ac.at/lbd.pac

Note: every time you change your proxy.pac file you need to click on "reload" to refresh the configuration.

Other browsers (including Chrome and Edge) use the default proxy config of your operating system. The default proxy does not work over system VPN connections. If you work on MacOS you can change the proxy config by going to Network -> WIFI (TU network) or VPN (TU VPN) -> Advanced -> Automatic Proxy Configuration. There you can insert the path of the proxy pac file as described above.

Test the connection

If you open the page http://c100.local:8088/cluster

you should get a list of Yarn applications.

Troubleshooting

In case you should get a "Dr.Who" error message, append ?user.name=username (where username is your login name) at the end of the URL, for instance:

http://c100.local:8088/cluster?user.name=username