Improving read performance
Be sure to review the page on Memtable Thresholds -- several important configuration and system settings are described there.
Lessen overall system impact of Compactions
To lower compaction priority (thus reducing its impact on the rest of the system, and making it take longer), add these options to cassandra.in.sh in 0.6.3 or later:
-XX:+UseThreadPriorities \ -XX:ThreadPriorityPolicy=42 \ -Dcassandra.compaction.priority=1 \
Useful JVM options
-XX:+UseCompressedOops # enables compressed references, reducing memory overhead on 64bit JVMs