Personal tools
You are here: Home Documentation Adding New Pools to dCache
Document Actions

Adding New Pools to dCache

by admin last modified 2007-07-24 09:36

This brief tutorial covers adding new pools to our Nebraska site.


dCache needs Sun's Java installed to work; consider this step #0. You can get the RPM from scientificlinux.org <!-- @page { size: 8.5in 11in; margin: 0.79in } P { margin-bottom: 0.08in } -->.  You could also grab the jdk rpm from another dcache pool that is already in service.  As a rule the rpm is left in root's home area after install.


After installing the Java JDK, replace the link from the system installed java to the new installed java.  On recent installs this entailed something like:

rm /usr/bin/java
ln -s /usr/java/jdk1.5.0_10/bin/java /usr/bin/java

We now deploy dcache pools using the vdt install package.  Grab the VDT install package from another dcache pool node.  The name is something like: vdt-dcache-SL4_64-1.0.7.tar.gz, this contains everything you need execpt the latest dcache RPMS.  Copy those over as well.  As of this writing it was: dcache-server-1.7.0-34.noarch.rpm.

Blow up the vdt tar ball and copy the rpms to the correct directory:

tar -xzvf vdt-dcache-SL4_64-1.0.7.tar.gz
cd vdt-dcache-SL4_64-1.0.7/RPMS
mv ~/dcache-server-1.7.0-34.noarch.rpm ./

Then install the RPM:
rpm -i dcache-server-1.7.0-34.noarch.rpm 
Now for the VDT magic.  Get the site-info.def file from a working dcache pool.  (Try to get the file from the last dcache installed pool so it up to date, or pull the master off of red, it should be in red's root home area or the /opt/kickstart area.)  This file contains the infomation for configuring the pool, so we'll have to edit one line.

Edit the site-info.def file to include the new pool.  Append the pool to the following line:

DCACHE_POOLS="dcache01.unl.edu:all:/mnt/raid/  dcache03.unl.edu:all:/mnt/raid/  dcache04.unl.edu:all:/mnt/raid/ dcache-s01.unl.edu:all:/mnt/raid/ dcache-s02.unl.edu:all:/mnt/raid1/ dcache-s02.unl.edu:all:/mnt/raid2/ dcache-s03.unl.edu:all:/mnt/raid/pool  dcache-s04.unl.edu:all:/mnt/raid dcache-s04.unl.edu:all:/opt thpc-1.unl.edu:all:/pool/pool dcache-fc01.unl.edu:all:/mnt/xz1 dcache-fc01.unl.edu:all:/mnt/xz2 dcache-s05.unl.edu:all:/mnt/raid dcache-s06.unl.edu:all:/mnt/raid dcache-s07.unl.edu:all:/mnt/raid dcache-s08.unl.edu:all:/mnt/raid dcache-s09.unl.edu:all:/mnt/raid new-pool.unl.edu:all:/mount/point_for/pool"
Once the site-info.def file is up to date, run the vdt install script:

./install.sh

This will place all you need to get dCache up and running in /opt/d-cache

If you aren't going to run a gridFTP door on the server you can skip down to "Start the server".

If we want to run a gridFTP door, there are some additional steps.
If we're running this node as a gridFTP door as well as a pool, then I'll assume you've installed the OSG:ce stack to manage the certificates. Then do the following:
a) sync the /etc/passwd and /etc/group files with a working gridFTP door.
b) create the mount point /pnfs/fs and edit the /etc/fstab to mount the /pnfs filesystem, the following line added to fstab should suffice:
dcache-pnfs.unl.edu:/pnfsdoors /pnfs/fs nfs hard,intr,ro,bg,noac,auto 0 0
c) create two links in the /pnfs area:
ln -s /pnfs/fs /pnfs/unl.edu
ln -s /pnfs/fs /pnfs/ftpBase
d) edit the node config file in /opt/d-cache/etc to turn on the gridFTP service.
vi /opt/d-cache/etc/node_config
<Go down to here>
#  ----  Services to be started on this node
#    The following services are only started on this node
#    if the corresponding parameter is set to 'yes'.
#    Exeption: The PnfsManager is started on the admin node
#              if the parameter is not specified.
GSIDCAP=no
DCAP=no
GRIDFTP=yes <----Change this line from no to yes
SRM=no
XROOTD=no
Follow all the commands for start the server with the added steps of repeating the process for dcache-core. 

Start the server:
[root@dcache01 ~]# /opt/d-cache/bin/dcache-pool start

Starting dcache pool: Starting dcache01Domain  6 5 4 3 2 1 0 Done (pid=23660)

Additionally, copy the pool scripts to the init directory and set it to start the service on boot.
[root@dcache01 ~]# cp /opt/d-cache/bin/dcache-pool /etc/init.d/dcache-pool
[root@dcache01 ~]# chkconfig --add dcache-pool
[root@dcache01 ~]# chkconfig dcache-pool on
To improve network performance modify the tcp settings to the following in /etc/sysct.conf:
# Kernel sysctl configuration file for Red Hat Linux
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.
# Controls IP packet forwarding
net.ipv4.ip_forward = 0
# Controls source route verification
net.ipv4.conf.default.rp_filter = 1
# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.kernel.core_uses_pid = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_sack = 0
net.core.rmem_max = 1048576
net.core.rmem_default = 87380
net.core.wmem_max = 131072
net.core.wmem_default = 32768
net.ipv4.tcp_rmem = 4096 87380 2097152
net.ipv4.tcp_wmem = 4096 32768 262144
net.ipv4.tcp_mem = 65536 87380 196608
vm.min_free_kbytes = 65536
vm.overcommit_memory = 2

Don't forget to add the pool to the resilentPools group.  Modify the /opt/d-cache/config/PoolManager.conf file on the dcache headnode.  There are three places the pool needs listed, register pool, default group and the resilient group.  Follow the examples already in that file.
Finally, check the usage info on the webpage: http:/dcache-head.unl.edu:2288/usageInfo


Powered by Plone, the Open Source Content Management System