Linux Command Examples

sed -n '10,20p' access.log
find /etc -name hadoop-env.sh 2>/dev/null

Create a file including 60 whoami
for n in $(seq 60)
do
echo 'whoami' >> whoami60
done
seq - print a sequence of numbers
SYNOPSIS
seq [OPTION]... LAST
seq [OPTION]... FIRST LAST
seq [OPTION]... FIRST INCREMENT LAST

Check shell type
echo $0
echo $SHELL

How Can I Use An "if" statement on a Single Command Line?
Basically, each element that *requires* a line after it must have a ; otherwise it shouldn't.
if [ 1 == 1 ]; then echo "true"; fi

Remove last character from file/string
sed -e 's/,$//' file

rev filename | cut -b 2- | rev
Basically, it reverses each line, cuts from the 2nd character to the end of line and reversing it back.

Find out commands that are not finished
We need check the log to find out commands that are not finished.
When command is started, it would print like: [date time] "Executing command: $CmdId".
when command is finished, it would print like: [date time] "Done executing command: $CmdId"

script(O(m*n)):
awk '/Executing command/ { print $5 }' log > executing-tmp
awk -F\" '{ print $1}' executing-tmp > executing ===> remove the last character ".

awk '/Done executing command/ { print $6 }' log > done

while read cmd
do
count=`grep -c $cmd done`
if [[ $count -eq 0 ]]; then
  print "$cmd was not finished";
  grep $cmd niCommandAgent.log
fi
done < "executing"

Find out files that are deleted but not released
When application forgets to close file stream, as long as application is running, System would be unable to release free up disk space held by deleted files.

When application is running, user deletes application logs, if not handled this case properly, - if not check file existence when writing log, - not close the already-deleted-files, it may cause system unable to release free up disk space held by deleted files.
This may cause big problems, such as application/system crashes because disk if full.

lsof > lsof.txt
COMMAND     PID       USER   FD      TYPE DEVICE     SIZE      NODE NAME

grep deleted lsof.txt | grep "3,1" | tr -s ' ' | cut -f6- -d' ' | sort -n -k2 | uniq
  3,1 84377 7365 /var/log/log (deleted)
   ^      ^      ^      ^
   |      |      |      +- Filename
   |      |      +-------- Inode # (unique ID inside the filesystem)
   |      +--------------- Size (bytes)
   +---------------------- Device ID (3,1)
 
The following script is used to how much disk space is occupied by unreleased files.

awk '/deleted/ && /3,1/ {total = total + $7} END { print "Totally space - " total/(1024*1024) " mb is not released" } ' lsof.txt
Post a Comment

Labels

Java (159) Lucene-Solr (110) Interview (61) All (58) J2SE (53) Algorithm (45) Soft Skills (36) Eclipse (34) Code Example (31) Linux (24) JavaScript (23) Spring (22) Windows (22) Web Development (20) Nutch2 (18) Tools (18) Bugs (17) Debug (15) Defects (14) Text Mining (14) J2EE (13) Network (13) PowerShell (11) Troubleshooting (11) Chrome (9) Design (9) How to (9) Learning code (9) Performance (9) UIMA (9) html (9) Http Client (8) Maven (8) Problem Solving (8) Security (8) bat (8) blogger (8) Big Data (7) Continuous Integration (7) Google (7) Guava (7) JSON (7) ANT (6) Coding Skills (6) Database (6) Scala (6) Shell (6) css (6) Algorithm Series (5) Cache (5) Dynamic Languages (5) IDE (5) Lesson Learned (5) Programmer Skills (5) Tips (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) Miscs (4) OpenNLP (4) Project Managment (4) Spark (4) System Design (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Bit Operation (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Firefox (2) Google Drive (2) Gson (2) How to Interview (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Python (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts