Visual Diff Tools for Developers

Diff Merge

Diff Merge is a great tool for any developer for Diffing and merging code. I have used it and found it to be in dispensable.

I would strongly recommend this tool. The company behind the tool is SourceGear, they are generous to distribute this gem for no cost to developers :). They also have other useful tools FORTRESS and VAULT, check it out at their website – @ www.sourcegear.com

Feature Highlights for time challenged :

• Diff Graphically shows the changes between two files. Includes intra-line highlighting and full support for editing.
• Merge Graphically shows the changes between 3 files. Allows automatic merging (when safe to do so) and full control over editing the resulting file.
• Folder Diff Performs a side-by-side comparison of 2 folders, showing which files are only present in one file or the other, as well as file pairs which are identical or different.
• Windows Explorer Integration. Right-click on any two files in Windows Explorer to diff them immediately.
• Configurable. Rulesets and options provide for customized appearance and behavior.
• International. Compatible with 42 different character encodings.
• Cross-platform. Identical feature set on Windows, Mac OS X, and Unix.

Cross Joins

Cross joins are used to return every combination of rows from two tables, this sometimes called a Cartesian product. In SQL Server you can use the CROSS JOIN keywords to define a cross join.

Cross joins can be cause out of memory issues in Big Data engines like spark when users are allowed to cross join on large datasets (i.e. millions of rows) as each partition has limited memory.

At the fundamental level Cross join can be thought of as nested for loop, where the code traverses thru the inner and outer join. The key issue will be how the memory is handled. If the code is good at allocating and deallocating memory at the inner loop ideally or at-least in the outer loop, the issues of OOM errors can be mitigated, however if we are relying on a external system like GC to manager memory, we can run into OOM issues.

for ( i = 0; i < 100M; i++)

{

allocate memory

for ( j=0; j < 100M; j++)

{

allocate memory

<Operations>

deallocate memory

}

deallocate memory

}

All about Browser Cache

There are 4 chrome caches, but predominantly we use 3 caches-
1. Memory cache
2. Http cache( or disk cache or prefetch cache)
3. Service worker ( only if service worker is enabled in the page, else not used).
4. Push cache ( http2 cache)
This was way back in 2017. However a lot has changed now and chrome now distinguishes disk cache and prefetch cache as separate but behind the scenes they use the same http cache implementation.
1. Memory cache: Browser memory or heap allocated to memory cache ( 0.16% of browser memory )
2. Disk cache: resources that are not accessed recently or not critical stored in disk  
3. Prefetch cache: Http Cache where any pre fetched resources are stored
4. Service worker ( if enabled, optional)
5. Push cache ( if enabled, optional)

Each resource is assigned to every resource, most critical resources are stored in the memory cache and the next critical cache are stored in http cache, if a resource link has a pre-fetch tag it will be automatically stored in http cache, 

If we store more things in http cache it is helpful to get it concurrently 

Ember Quick Start

Installing Ember: 

1. Install HomeBrew or NPM :
   1. HomeBrew: ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”
   2. NPM:

      sudo PATH=”$PATH:/usr/local/bin” /usr/local/bin/npm npm install npm@latest -g

2. Install Node:
3. Install Ember:

   sudo PATH=”$PATH:/usr/local/bin” /usr/local/bin/npm install -g ember-cli@2.17

4. Run Ember test server to test if it works: change to ember_kickstart folder and run

   sudo PATH=”$PATH:/usr/local/bin” /usr/local/lib/node_modules/ember-cli/bin/ember serve

Testing Websites on Webpage test

1. Open your website with logged in experience. Look at network waterfall, copy text of cookie from the base page request headers.

2. Save the following as a genWPTScript.sh file (update it to use your assigned website instead of netflix.com):

tr ‘;’ ‘\n’ | awk ‘{print “setCookie\thttps://www.netflix.com\t”, $1}’;

echo “navigate https://www.netflix.com“

3. In shell, run:

echo “<copied cookie from step 1>” | sh getWPTScript.sh

4. Copy output and paste in the script section of webpagetest.org, make sure you mark script is sensitive, then run the test

Using ADB commands to work with android apps

//start the app

adb shell am start -n com.myapp.android/.packagename.LaunchActivity

//send to background

adb shell am start com.android.calculator2

//kill the app

adb shell am force-stop com.myapp.android

Test DOC

document-1020383371

Memory Mapped Files in Java

Important Facts about Memory Mapped files in Java – thanks to Java Revisited @ Blogspot

1. Java supports Memory mapped IO with java.nio package.
2. Memory mapped files is used in performance sensitive application, e.g. high frequency electronic trading platforms.
3. By using memory mapped IO, you can load portion of large files in memory.
4. Memory mapped file can result in page fault if requested page is not in memory.
5. Ability to map a region of file in memory depends on addressable size of memory. In a 32 bit machine, you cannot access beyond 4GB or 2^32.
6. Memory mapped IO is much faster than Stream IO in Java.
7. Memory used to load File is outside of Java heap and reside on shared memory which allows two different processes to access File.
8. Reading and writing on memory mapped file is done by operating system, so even if your Java program crashes after putting content into memory, it will make to File until OS is fine.
9. Prefer Direct Byte buffer over Non Direct Buffer for faster performance.
10. Don’t call MappedByteBuffer.force() method too often, this method is meant to force operating system to write content of memory into disk, So if you call force() method each time you write into memory mapped file, you will not see true benefit of using mapped byte buffer, instead it will be similar to disk IO.
11. In case of power failure or host failure, there is slim chance that content of memory mapped file is not written into disk, which means you could lose critical data.

The Core of Microprocessors

Most modern computers today operate with multiple processors and the processor in-turn have multiple cores. Below is a breakdown of the architecture of a modern computer / processor. I took these notes so that it’s easy for me to understand and I hope it helps others.

Motherboards have  sockets that can fit Microprocessors  ( usually 2 )

Each Microprocessor has multiple computing components, which are actual independent central processing units also known as “Cores” ( usually 6 per socket ) . These cores share some computing functions like floating point calculations, while performing integer calculations independently.

Intel introduced the concept of Hyper-Threading. For each processor core that is physically present, the operating system addresses two virtual or logical cores, and shares the workload between them when possible.  ( usually there are 2 Hyper-Threads per core ).

When you issue the command like “procinfo” to list the number of CPU’s in a server (Linux/Unix ) . The number of CPU might be misleading as the command identifies each Hyper-Thread as a core. Hence A server with 2 sockets and 6 core processor in each socket will be reported to have 24 CPU’s. This is correct from OS perspective as OS treats each Hyper-Thread as a CPU.

Depending on the hardware architecture there can be multiple levels of caches based. The block diagram below describes the association of cache/memory  to processing units in a server.

 

 

Linux Performance Analysis & Tools

Linux Performance Analysis and Tools from brendangregg

GC metrics you should care

• gen0(s)     – young gen collection time, excluding gc_prologue & gc_epilogue.
• gen0t(s)    – young gen collection time, including gc_prologue & gc_epilogue
• gen0usr(s)  – young gen collection time in cpu user secs
• gen0sys(s)  – young gen collection time in cpu sys secs
• gen1i(s)    – train generation incremental collection
• gen1t(s)    – old generation collection/full GC
• cmsIM(s)    – CMS initial mark pause
• cmsRM(s)    – CMS remark pause
• cmsRS(s)    – CMS resize pause // rarely look. Probably we set OG at fixed size
• GC(s)       – all stop-the-world GC pauses
• cmsCM(s)    – CMS concurrent mark phase   // don’t care
• cmsCP(s)    – CMS concurrent preclean phase // don’t care
• cmsCS(s)    – CMS concurrent sweep phase // don’t care
• cmsCR(s)    – CMS concurrent reset phase // don’t care
• alloc(MB)   – object allocation in MB (approximate***)
• promo(MB)   – object promotion in MB (approximate***)
• used0(MB)   – young gen used memory size (before gc)
• used1(MB)   – old gen used memory size (before gc)
• used(MB)    – heap space used memory size (before gc) (excludes perm gen)
• commit0(MB) – young gen committed memory size (after gc)
• commit1(MB) – old gen committed memory size (after gc)
• commit(MB)  – heap committed memory size (after gc) (excludes perm gen)
• apptime(s)  – amount of time application threads were running
• appstop(s) – amount of time application threads were stalled
• safept(s)   – amount of time the VM spent at safepoints (app threads stopped)