Go to previous page / next page of Tier3 site log

03. 01. 2013 dCache file deletion

File Deletion Test

with old file (i.e. short PNFSID)

0002000000000000069DE5B0 => /pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root

In the Chimera database

chimera=> SELECT * FROM t_locationinfo WHERE ipnfsid = '0002000000000000069DE5B0';
         ipnfsid          | itype |  ilocation   | ipriority |            ictime             |            iatime             | istate 
--------------------------+-------+--------------+-----------+-------------------------------+-------------------------------+--------
 0002000000000000069DE5B0 |     1 | t3fs14_cms_3 |        10 | 2012-11-30 08:55:27.410752+01 | 2012-11-30 08:55:27.410752+01 |      1
(1 row)

And on disk

[root@t3fs14 data]# ls -lah /mnt/data3/t3fs14_cms/pool/data/0002000000000000069DE5B0
-rw-r--r-- 1 root root 302M Nov 30 12:11 /mnt/data3/t3fs14_cms/pool/data/0002000000000000069DE5B0

User interaction with the file

lcg-ls -l 'srm://t3se01.psi.ch:8443/srm/managerv2?SFN=/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root'
-rw-r--r--   1     2     2 315977413               ONLINE /pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root
	* Checksum: d8fe3550 (adler32)
	* Space tokens: 5438474
lcg-del -l 'srm://t3se01.psi.ch:8443/srm/managerv2?SFN=/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root'
lcg-ls -l 'srm://t3se01.psi.ch:8443/srm/managerv2?SFN=/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root'
/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root: [SE][Ls][SRM_INVALID_PATH] could not get storage info by path: No such file or directory ///pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile.root

New database query

chimera=> SELECT * FROM t_locationinfo WHERE ipnfsid = '0002000000000000069DE5B0';
 ipnfsid | itype | ilocation | ipriority | ictime | iatime | istate 
---------+-------+-----------+-----------+--------+--------+--------
(0 rows)

But on the file server

[root@t3fs14 data]# ls -lah /mnt/data3/t3fs14_cms/pool/data/0002000000000000069DE5B0
-rw-r--r-- 1 root root 302M Nov 30 12:11 /mnt/data3/t3fs14_cms/pool/data/0002000000000000069DE5B0

with new file (i.e. long PNFSID)

0000EE8D339201A34DBD9E1A8DDD45FE1BD0 => /pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root

the same steps as above

chimera=> SELECT * FROM t_locationinfo WHERE ipnfsid = '0000EE8D339201A34DBD9E1A8DDD45FE1BD0';
               ipnfsid                | itype |  ilocation   | ipriority |           ictime           |           iatime           | istate 
--------------------------------------+-------+--------------+-----------+----------------------------+----------------------------+--------
 0000EE8D339201A34DBD9E1A8DDD45FE1BD0 |     1 | t3fs11_cms_1 |        10 | 2012-11-30 12:12:45.008+01 | 2012-11-30 12:12:45.008+01 |      1
(1 row)

on t3fs11

# ls -lah /data1/t3fs11_cms_1/data/0000EE8D339201A34DBD9E1A8DDD45FE1BD0
-rw-r--r--   1 root     root        301M Nov 30 12:12 /data1/t3fs11_cms_1/data/0000EE8D339201A34DBD9E1A8DDD45FE1BD0

lcg-ls -l 'srm://t3se01.psi.ch:8443/srm/managerv2?SFN=/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root'
-rw-r--r--   1     2     2 315977413               ONLINE /pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root
	* Checksum: d8fe3550 (adler32)
	* Space tokens: 5980084
lcg-del -l 'srm://t3se01.psi.ch:8443/srm/managerv2?SFN=/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root'
lcg-ls -l 'srm://t3se01.psi.ch:8443/srm/managerv2?SFN=/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root'
/pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root: [SE][Ls][SRM_INVALID_PATH] could not get storage info by path: No such file or directory ///pnfs/psi.ch/cms/trivcat/store/user/dmeister/tmp/testfile3.root

chimera=> SELECT * FROM t_locationinfo WHERE ipnfsid = '0000EE8D339201A34DBD9E1A8DDD45FE1BD0';
 ipnfsid | itype | ilocation | ipriority | ictime | iatime | istate 
---------+-------+-----------+-----------+--------+--------+--------
(0 rows)

# ls -lah /data1/t3fs11_cms_1/data/0000EE8D339201A34DBD9E1A8DDD45FE1BD0
-rw-r--r--   1 root     root        301M Nov 30 12:12 /data1/t3fs11_cms_1/data/0000EE8D339201A34DBD9E1A8DDD45FE1BD0

Conclusion

The files are 'deleted' correctly from the database but the actual file on the file server is not removed. To clean this up we can run pnfs register on the corresponding storage pool

[t3dcachedb04.psi.ch] (local) admin > cd t3fs14_cms_3
[t3dcachedb04.psi.ch] (t3fs14_cms_3) admin > pnfs register

which has the desired effect

[root@t3fs14 data]# ls -lah /mnt/data3/t3fs14_cms/pool/data/0002000000000000069DE5B0
ls: cannot access /mnt/data3/t3fs14_cms/pool/data/0002000000000000069DE5B0: No such file or directory

The dCache logfiles do not show anything useful...

Update from 2013-02-11

A second test after the upgrade to dCache v1.9.12 shows that file removal is now working correctly; the average delay is only about 120s.

-- DanielMeister - 2013-01-03

Go to previous page / next page of Tier3 site log