Linux, Splunk

How to install Splunk Enterprise on Linux

by jb

Basic Installation

These are the steps to install the 60 days trial version of Splunk Enterprise 9.4.0 on RedHat Enterprise Linux 8.1. I used Oracle VirtualBox as the virtualization software running on Windows 10. First we create an account on splunk.com and download the Linux rpm file via wget. As the root user we install the rpm file as follows:

rpm -i splunk-9.4.0.x86_64.rpm

[root@lin Downloads]# rpm -i splunk-9.4.0.x86_64.rpm
warning: splunk-9.4.0.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID b3cd4420: NOKEY
no need to run the pre-install check
complete
[root@lin Downloads]#

The installation created a user and a group named splunk. We change the password of the splunk user as follows:

passwd splunk << EOF
splunk
splunk
EOF

Changing password for user splunk.
New password: BAD PASSWORD: The password is shorter than 8 characters
Retype new password: passwd: all authentication tokens updated successfully.
[root@lin ~]#

Now we start splunk for the first time. Enter admin as the username and choose a password:

su - splunk
cd /opt/splunk/bin/
./splunk start --accept-license

[root@lin Downloads]# su - splunk
[splunk@lin ~]$ cd /opt/splunk/bin/
[splunk@lin bin]$ ./splunk start --accept-license

This appears to be your first time running this version of Splunk.

Splunk software must create an administrator account during startup. Otherwise, you cannot log in.
Create credentials for the administrator account.
Characters do not appear on the screen when you type in credentials.

Please enter an administrator username: admin
Password must contain at least:
   * 8 total printable ASCII character(s).
Please enter a new password:
Please confirm new password:
Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'.
Generating RSA private key, 2048 bit long modulus
..+++++
...........................................................+++++
e is 65537 (0x10001)
writing RSA key

Generating RSA private key, 2048 bit long modulus
.......+++++
.................................+++++
e is 65537 (0x10001)
writing RSA key

Moving '/opt/splunk/share/splunk/search_mrsparkle/modules.new' to '/opt/splunk/share/splunk/search_mrsparkle/modules'.

Splunk> Now with more code!

Checking prerequisites...
        Checking http port [8000]: open
        Checking mgmt port [8089]: open
        Checking appserver port [127.0.0.1:8065]: open
        Checking kvstore port [8191]: open
        Checking configuration... Done.
                Creating: /opt/splunk/var/lib/splunk
                Creating: /opt/splunk/var/run/splunk
                Creating: /opt/splunk/var/run/splunk/appserver/i18n
                Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css
                Creating: /opt/splunk/var/run/splunk/upload
                Creating: /opt/splunk/var/run/splunk/search_telemetry
                Creating: /opt/splunk/var/run/splunk/search_log
                Creating: /opt/splunk/var/spool/splunk
                Creating: /opt/splunk/var/spool/dirmoncache
                Creating: /opt/splunk/var/lib/splunk/authDb
                Creating: /opt/splunk/var/lib/splunk/hashDb
                Creating: /opt/splunk/var/run/splunk/collect
                Creating: /opt/splunk/var/run/splunk/sessions
New certs have been generated in '/opt/splunk/etc/auth'.
        Checking critical directories...        Done
        Checking indexes...
                Validated: _audit _configtracker _dsappevent _dsclient _dsphonehome _internal _introspection _metrics _metrics_rollup _telemetry _thefishbucket history main summary
        Done
        Checking filesystem compatibility...  Done
        Checking conf files for problems...
        Done
        Checking default conf files for edits...
        Validating installed files against hashes from '/opt/splunk/splunk-9.4.0-6b4ebe426ca6-linux-amd64-manifest'
        All installed files intact.
        Done
All preliminary checks passed.

Starting splunk server daemon (splunkd)...
Generating a RSA private key
......+++++
..............................+++++
writing new private key to 'privKeySecure.pem'
-----
Signature ok
subject=/CN=lin.fritz.box/O=SplunkUser
Getting CA Private Key
writing RSA key
PYTHONHTTPSVERIFY is set to 0 in splunk-launch.conf disabling certificate validation for the httplib and urllib libraries shipped with the embedded Python interpreter; must be set to "1" for increased security
Done
                                                           [  OK  ]

Waiting for web server at http://127.0.0.1:8000 to be available..................... Done


If you get stuck, we're here to help.
Look for answers here: http://docs.splunk.com

The Splunk web interface is at http://lin.fritz.box:8000

[splunk@lin bin]$

Now we activate systemd to start splunk at boot time:

/opt/splunk/bin/splunk stop
exit
/opt/splunk/bin/splunk enable boot-start -systemd-managed 1 -user splunk
init 6

[splunk@lin bin]$ /opt/splunk/bin/splunk stop
Stopping splunkd...
Shutting down.  Please wait, as this may take a few minutes.
...                                                        [  OK  ]
Stopping splunk helpers...
                                                           [  OK  ]
Done.
[splunk@lin bin]$ exit
logout
[root@lin Downloads]# /opt/splunk/bin/splunk enable boot-start -systemd-managed 1 -user splunk
Systemd unit file installed at /etc/systemd/system/Splunkd.service.
Configured as systemd managed service.
[root@lin Downloads]# init 6

After the server restart we see that the Splunkd service is running:

systemctl status Splunkd
[root@lin ~]# systemctl status Splunkd
● Splunkd.service - Systemd service file for Splunk, generated by 'splunk enable boot-start'
   Loaded: loaded (/etc/systemd/system/Splunkd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2024-12-29 17:55:30 CET; 5min ago
  Process: 1138 ExecStartPost=/bin/bash -c chown -R splunk:splunk /sys/fs/cgroup/memory/system.slice/Splunkd.service (code=exited, status=0/SUCCESS)
  Process: 1055 ExecStartPost=/bin/bash -c chown -R splunk:splunk /sys/fs/cgroup/cpu/system.slice/Splunkd.service (code=exited, status=0/SUCCESS)
 Main PID: 1054 (splunkd)
    Tasks: 166 (limit: 74402)
   Memory: 1.2G (max: 11.4G)
   CGroup: /system.slice/Splunkd.service
           ├─1054 splunkd --under-systemd --systemd-delegate=yes -p 8089 _internal_launch_under_systemd
           ├─2790 [splunkd pid=1054] splunkd --under-systemd --systemd-delegate=yes -p 8089 _internal_launch_under_systemd [process-runner]
           ├─3972 compsup daemon
           ├─4023 /opt/splunk/var/run/supervisor/pkg-run/pkg-identity4241153237/identity
           ├─4027 /opt/splunk/var/run/supervisor/pkg-run/pkg-agent-manager2026604619/agent-manager
           ├─4070 /opt/splunk/bin/splunkd instrument-resource-usage -p 8089 --with-kvstore
           ├─4125 /opt/splunk/bin/python3.9 /opt/splunk/etc/apps/splunk_secure_gateway/bin/ssg_alerts_ttl_modular_input.py
           ├─4137 /opt/splunk/bin/python3.9 -O /opt/splunk/lib/python3.9/site-packages/splunk/appserver/mrsparkle/root.py --proxied=127.0.0.1,8065,8000
           └─4139 /opt/splunk/bin/python3.9 /opt/splunk/etc/apps/splunk_secure_gateway/bin/ssg_enable_modular_input.py

Dec 29 17:55:38 lin.fritz.box splunk[2446]:                 Validated: _audit _configtracker _dsappevent _dsclient _dsphonehome _internal _introspection _metrics _metrics_rollup _telemetry _thefishbucket history main summary
Dec 29 17:55:38 lin.fritz.box splunk[2446]:         Done
Dec 29 17:55:41 lin.fritz.box splunk[1054]:         Checking filesystem compatibility...  Done
Dec 29 17:55:41 lin.fritz.box splunk[1054]:         Checking conf files for problems...
Dec 29 17:55:41 lin.fritz.box splunk[1054]:         Done
Dec 29 17:55:41 lin.fritz.box splunk[1054]:         Checking default conf files for edits...
Dec 29 17:55:41 lin.fritz.box splunk[1054]:         Validating installed files against hashes from '/opt/splunk/splunk-9.4.0-6b4ebe426ca6-linux-amd64-manifest'
Dec 29 17:55:42 lin.fritz.box splunk[1054]: PYTHONHTTPSVERIFY is set to 0 in splunk-launch.conf disabling certificate validation for the httplib and urllib libraries shipped with the embedded Python interpreter; must be set to "1" for increased security
Dec 29 17:55:42 lin.fritz.box splunk[1054]: 2024-12-29 17:55:42.694 +0100 splunkd started (build 6b4ebe426ca6) pid=1054

At that point I hit an error where the mongod process was not able to start and the log showed:

mongod exited abnormally (exit code 4, status: PID 34135 killed by signal 4: Illegal instruction)

I solved the problem by applying the steps described here. After applying the fix the service starts without error. The desktop / mobile web interface can be accessed with the URL: http://<hostname>:8000 using the username: admin and the password specified during the installation:

Installation of Splunk Mobile

If you want to use the mobile app Splunk Mobile you need to perform some additional tasks. First on the Apps section on the main page select “Splunk Secure Gateway”. I selected the defaults and choose Europe (Central) as the Spacebridge Location as this had the fastest response time for my location. Then select splunk mobile and click next:

A QR code shows up. Now install Splunk Mobile on your mobile and select Sign In and click on the + next to Private Instances. In the Code tab you can select Open Camera to scan this QR code. Thats it. We now have a basic splunk installation running on Linux and the possibility to login with our Desktop or with the mobile app.