How to Improve Design Skills


How to Improve Design Skills




How to Improve Design Skills

Jeffery Yuan
Updated at June 16, 2019

Agenda

  • How to Design
  • System Design Principles
  • Learning from Open Source
  • Learning from Existing Products
  • System Design Practices

How to Design

How to Design

  • Take time to think about your design
    • Minimize upfront design or YAGNI
    • It doesn’t mean you don’t take time to design the component
  • Components related
  • Impact to other components
  • What are alternatives?
  • Welcome different approaches and discussion

How to Design

  • Estimation
    • back-of-the-envelope calculation
  • Estimated data size, QPS
  • Take time to design data schema
    • As it’s difficult to change them after deploy to prod
  • Better user experience
    • Thinking from client/user perspective
    • How they use it, what they would like to know

Reflection – Lesson Learned

Reflection – Lesson Learned

  • What mistakes we made
    • Where to store data: dynamodb or not?
    • The key for Solr schema
  • Why they happened:
  • Not consider near-future requirements
  • Make decisions carelessly

Reflection – Lesson Learned

  • Better client library
    • Only contains library and code that client need
  • Package shared configuration in the library

System Design Principles

System Design Principles

  • Idempotent
  • Policy to expire/archive data - Less data
  • Optimize data for read
    • Denormalization
  • Read Heavy vs Write Heavy
  • Design to Be Disabled - feature toggle
  • Isolate Faults - Circular breaker
  • Throttling - Rate limit

System Design Principles Cont.

  • Stateless
  • Asynchronous
    • Back pressure with exponential backoff
  • Message queues
  • Cache
  • Visibility – monitoring
  • Separation of concerns
    • Separate read and write

System Design Principles Cont.

  • CAP
  • Graceful Degradation
  • Be Robust - Hide error as much as possible
  • Be conservative in what you send, be liberal in what you accept
  • Make your apps do something reasonable even if not all is right

Learning from Open Source

Learning from Open Source

  • What makes them popular
  • When to use them, when not

Cassandra

  • LSM(Log Structured Merge Trees)
    • append-only
  • SSTable
  • MemTable - SSTable in memory
  • How C* handles delete: Tombstone(grace period)
  • Merkle trees
  • Bloom Filter
  • Index
  • CommitLog

Cassandra Cont.

  • Serialize cache data (row-cache, key cache) to avoid cold restart
  • Session Coordinator
  • Gossip protocol
  • Seed nodes
  • Consistent Hashing
  • Eventual Consistency
    • W+R > N
  • Local Index (vs Global Index)

Kafka

  • Why it is fast
  • Sequentially read/write vs random read/write
  • Memory Mapped File
  • Zero copy
  • Batch data(compressed)
  • Partition: ordered, immutable, replicated
  • Consumer group

Database

  • Sharding
  • Replication
  • Master/Slave, Multi-master

Learning from Existing Products

  • Twitter/FB timeline
  • Pull/Push/Mixed Model
  • FB Haystack/Photo storage

System Design Practices

System Design Practices

  • URL shortener
    • read heavy
    • able to disable write functions
  • Design key-value store
  • Crawler
    • Re-crawling
    • cur+2t or cur+t/2 based on changed or not
  • Design search engine
    • In-memory version: Data structure
    • Distributed: Solr Cloud internal design

System Design Practices

  • Design score/rank system for social game
  • Search nearby places: GeoHash
  • Design Chat app
  • Design logging collection and analysis system
  • Design shopping cart
    • guest cart
  • Design Hit Counter
  • Design rate limiter
  • Design Miao Sha

Resource

Resource

  • Designing Data-Intensive Applications
  • Scalability Rules: Principles for Scaling Web Sites
  • The Art of Scalability

Resource Cont.

How to Improve Your Skills as a Programmer




How to Improve Your Skills as a Programmer

How to Improve Your Skills as a Programmer

Jeffery Yuan

Updated at June 16, 2019

Skills that are important for engineers

  • Problem Solving
  • Coding
  • Design
  • Eagerness to learn
  • Focus on soft skills in this presentation

Learn from others

  • Be humble and open-minded
    • If others are doing things different, use different tools, anything that you don’t know
    • Ask, learn and try them
  • Learn the knowledge itself
  • But also learn their thinking process
  • Ask help for same/similar things once, then you can do it

Learn how to do daily work better

Learn how to do daily work better

  • How to write clean and bug-free code
  • Source of common bugs
  • Learn from Bugs/Mistakes we made

Learn how to do daily work better Cont.

Learn the things you use

Learn the tools you use everyday

Learn the tools you use everyday

Mentoring/teaching

  • Improve your skills:
    • quick thinking, trouble shooting on spot
  • Save your time
    • don’t only tell the solution, but how you analyze the problem
  • Grow as a team/together
  • Good for the team

Share your knowledge

  • Take time to reflect and write down
  • you will have a better understanding
  • Create the environment that people share knowledge
    • you can also learn from others
  • Share formally or informally
    • slack, email, or group discussion
    • team-tech-only slack

Communication Skills

  • Listen before Ask/Talk
  • How to ask question/help
  • Be helpful
  • Keep calm and focused

What to learn

  • Trending(Atom)
  • Common tech used
    • Kafka/C*/Solr etc
  • How Big Tech Companies Build Things

Java Optional Best Practices


Linux File Manager Nemo Tips and Tricks


Spell and Grammar Check for Markdown


Labels

adsense (5) Algorithm (69) Algorithm Series (35) Android (4) ANT (6) bat (8) Become a Better You (4) Big Data (7) Blogger (14) Bugs (4) Cache (5) Chrome (17) Code Example (29) Code Quality (6) Coding Skills (5) Concurrency (4) Database (7) Debug (16) Design (5) Dev Tips (62) Eclipse (32) GAE (4) Git (5) Good Programming Practices (4) Google (27) Guava (7) How to (9) Http Client (8) IDE (6) Interview (88) J2EE (13) J2SE (49) Jackson (4) Java (177) JavaScript (27) JSON (7) Learning code (9) Lesson Learned (6) Linux (22) Lucene-Solr (112) Mac (10) Maven (8) Memory Usage (4) Network (9) Nutch2 (18) OpenNLP (4) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) regex (5) Review (4) Scala (6) Security (9) Soft Skills (38) Spark (4) Spring (22) System Design (11) Testing (6) Text Mining (14) Tips (12) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) xml (5)

Trending