OK, this feels too easy, and so I'm super paranoid about it. Anybody see anything wrong with this? I don't want to blow some of this stuff up.


ARCHIVE_DATE=`date +%Y%m%d`

find /sftp/* -type f -mtime +365 | while read LINE
tar -rf ${ARCHIVE_LOCATION}/archive_${ARCHIVE_DATE}.tar "${LINE}"
rm -f "${LINE}"

gzip -9 ${ARCHIVE_LOCATION}/archive_${ARCHIVE_DATE}.tar

You can remove "Anti-snobbishness and humbleness is more fun." from your description.

@The_Quantum_Alpha @mike ah, the real face is showing, petty and haughty, and then begging for money, yeah...

@Ashy_Mini @StrawberryShortcake @sotolf @mike it's alright!🙃
Shouldn't I deserve this for my answer? Was not the kindest, but I have my reasons to provide that one.

Hater gonna hate!

I'm now well secured financially, and I know I'm not what he said, so I'm not offended at all!😊

No, Sorolf, you won't have my job, if that's what you're jealous of !🤭

I have wonderful friends, and I'm happy !

@StrawberryShortcake @The_Quantum_Alpha @mike didn't start it ;) I'm mostly nice, but some times a sharp comment is fitting.

I'm still not sure these are not just clever bots.

@mike I would create a few dummy files to test it before running, changing the "rm" with an "echo", just to make sure the commands are correct.

@mike if the file names contain quotes, you could enable rce (assuming you can’t trust the file names; if you created all of them, it should be fine)

also, currently, this will delete the original whether it was compressed successfully or not. try using a `&& rm -f ...` or `if [ $? = 0 ]; do rm -f ...; done` to preserve the original if the archive fails. you could also use the `--remove-files` flag to do it all in one line.)

@mike then you could make it a one liner by combing the compression step, ie `tar czvf`
and then you could use xargs instead of while read line (and you could pass maxprocs to parallelize the process!

@Spaceface16518 @mike I would not use the remove file option... It's dangerous if tar fails in the middle of compression. You have to get already compressed files out of the archive again for a complete retry.
It works like compress file one --> remove file one... Compress file n --> remove file n...

@BollerwagenPicard @mike i agree now that i know this behavior, but it’s still better than removing the file whether or not it was compressed. at least the tar still contains the files that did succeed.
but yeah separating the rm step might be useful, like someone else mentioned.

@mike maybe set -e so that it stops on error, if that works on ksh, I also use -u by default.

System could crash in whatever way while archiving. Someone would maybe create an archive dir and move things there, tar to a temporary file, move temporary archive file to a final name and remove the archive dir.

Sign in to participate in the conversation

Fosstodon is an English speaking Mastodon instance that is open to anyone who is interested in technology; particularly free & open source software.