TIL, 2017-06-23

Keynote, Emily Stolfo

• Cathedral thingie which was made by several dudes.
• Studying the structure of mongoid (10 years old).
• Restoring Mongoid:
  • Study the structure, identify weaknesses
  • Refactoring
  • Responsible maintenance
• Mongoid structural analysis:
  • Group issues into categories.
  • Step through major code paths, and take notes on those parts. Because there was a lot of metaprogramming, this was sort of hard.
  • Weakness identified.
• Refactoring: Not “rehactoring”. Is this a healthy change?
• Refactor one piece at a time.
• Responsible maintenance: send updates when you change stuff.
• Measuring entropy: How difficult is it to make changes? Can you explain the design? How is performance?
• “What am I going to do today to make Mongoid better?”

To Code is Human, Don Werve

• Software can only be as healthy as the engineers that create it.
• Manage your energy, not your time. We all know the zone. Imagine if most if not all of your hours are like that.
• Exercise: whatever you like.
• Food: basically anything.
• Human working memory: 7 +- 2 items at a time. This is why we have rules like character limits, function line limits.
• We have evolved to remember stories. For millions of years, we shared experiences with each other in story form, not in written form.
• Code that reads like well-written prose gets a cognitive fast-track for both comprehension and recall.
• You can make a finite number of decisions each day, use them wisely (decision fatigue).
• Prison parole: you really want to go there early so the judges aren’t tired yet. So they will have better decision making.
• The reward cycle is why we do everything and anything.
• Human speech is massively distracting because we have evolved to focus on it.
• Pair programming makes an open office tolerable, why? Because you get an instant code review.

Shitlist-driven development, Florian Weingarten

• The Shopify monolith: > 400k shops, multi-tenant architecture
• 20k-40k requests per second
• 800 contributors (developers/designers)
• Everybody can merge to master and deploy to production. 40-50 deploys (50-100 PRs) shipped to production per day.
• If you want small and often , you need to deploy fast.
• Deploys have to be shorter than 10 minutes, since you deploy x times in a day, there is a limit.
• Deploy bottleneck: speed.
  • Parallel CI builds
  • Build containers in advance and quickly
  • Avoid booting application multiple times during container builds.
  • Deploy to many servers in parallel.
  • Basically, reduce application boot time.
  • Reduce application shutdown time (ex: if you’re using Unicorn, you have to wait for it to finish).
• Deploy bottleneck: Humans
  • Asking ops team to deploy doesn’t scale.
  • Asking people when a good time to deploy is doesn’t scal.
  • Asking everyone to pay a ttention to errors during a deploy doesn’t scale.
  • Asking developers to deploy themselves doesn’t scale.
  • Basically humans don’t scale.
• There should be a non-human tool.
  • When CI passes, deploy to prod.
  • Revert commits if something breaks (not sure if I got this correctly).
• Slack channel to see if the code is going out. We don’t want commits to pile up. If the application gets really big, educating people is something that you can automate.

Monolith at Scale: Too Many Cooks in the Kitchen

• Deprecating methods: what if someone keeps on reverting a deprecation message, you can’t just email or slack everyone if there are 800 of you.
• What you can do–raise an error, but you have to make sure that some other methods can still call the deprecated one. Something like raise "error" unless shitlisted?(people_who_can_call_this_method)
• So basically do an around_action or around_perform for controllers/jobs, showing which controller actions can trigger this.
• When people fix their part, they can remove their controller from the shitlist, so you show some element of progress.
• Error message: Say what is wrong, say why it is deprecated, give a possible solution, and a way to contact the writers.

Tests

• Unreliable test: on the same version of the code, the test sometimes passes and sometimes fails.
• 750 runs per day, 70k lines, if we fail 1% of the time we lose 1 hour of productivity per day.
• Flaky test: a time-dependent, load-dependent test.
• Leaky test: order-dependent test.
• When you identify which tests are problematic–if you think this is flaky, you can do a grind, you run the single test in a container about 100 times.
• Bisect test–trying to figure out which tests run before the test that leaks. This is because there might be a test that mutates the application state, causing subsequent tests to fail.
• Annoying tests: caching, autoloading, global state, mutation, transactional fixtures, if you do an ALTER TABLE, it doesn’t revert the database changes. (lol)

Spinning up micro-services using Ruby/Kafka, Ankita Gupta

• Event stream: If you have a user creation service, you can the push the event out to an event bus which sends out emails or notifications.
• Kafka: a distributed, partitioned, replicated commit log service.
• When publishing, you want to have some contract when sending things over to it.
• Karafka: Something. IDK lol.
• Pheromone:

  publish [
    {
      event_types: [:create],
      topic: :topic1,
      message: ->(obj) { { name: obj.name } }
    }
  ]

One Way to Encourage the Open Source Community, Takayuki Matsubara

• Github stars: appreciation, support, or a “favorite projects” list.
• Usually we do not acknowledge gems that are the dependencies of other gems.
• bundle-star: put a star on Github gems your gem depends on.

Plan of action: we need more women in programming!, Marion Schleifer

• Reputation: think that programming is still male? (didn’t hear)
• People think they have to be ridiculously intelligent about a programmer, due to Hollywood?
• “Everyone has the potential to be a great developer, but not everyone has the opportunity.”
• If a talented programmer doesn’t become one because they think they aren’t supposed to, that’s a loss.
• Just tell people that programming is a very creative job.
• Provide opportunities because you can.
• What we can do: go to schools, or if not, then just show people your code.

One Blind Weekend, Jinny Wong

• No binoclar vision, inaccurate perception of depth, and 3D glasses are just green.
• Screen readers.
• Semantic HTML: If not a header, not use it, because screen readers categorize based on <h1> to <h6>.
• Consider things like role="navigation", <section> since screen readers use it.
• Give context: A screen user won’t know what an input field is for unless you provide context using label.
• Make sure your pictures have alt text, the screen reader will say its an imge already.
• Hover menus, make sure the element can still be tabbed to.
• Infinite scrolling: add a load more button, or standard pagination.
• Link should a <a>, not something with a click handler.
• meta name="viewport".
• 10% of males are color blind–don’t depend solely on colors to convey context.
• Microsoft Inclusive Design.

Keynote, Akira Matsuda

• Ruby refinements are weird.
• protected scope.
• In the Rails codebase, almost everyone is using protected but they should change it to private.
• private def: You can do private def method end, to show if it is a private method.
• git grep
• Keyword arguments.
• Encoding in Ruby, nobody uses non-UTF templates except in Japan.
• Propose to remove AS::Multibyte.
• fstring literal.

Goldilocks and the Three Code Reviews, Vaidehi Joshi

• LGTM = looks good to me. Jesus Christ why do I not know this.
• Collective ownership in construction–this is what code reviews are meant to do.
• Code Complete.

• Inspections, defect-detection, not correction. (not listening at this part)

• Short reviews: Small changes (< 5 lines?) tend to be more error-prone than larger changes (???). Teams with short code reviews move from 55% error rate to 2%.
• Walk-through: (More popular form.) 2 engineers talk about the code that was changed, it was helpful, but not the most effective. They are a great learning opportunity to learn shit.
• Peer Reviews in Software: A Practical Guide. Were the code reviews she was part in actually useful?
• Surveyed developers on code reviews.
  • Most people think that code reviews made them a better developer.
  • Swift developers the most, but generally all.
  • There were still 10% of people that had to ask for a code review.
  • Doesn’t really matter what language you are using, what matters is the people who are working on the thing.
  • Most (64%) had just one reviewer. However, 62% spent 5-20 minutes reviewing, and only 38% received the feedback within hours (20% within one day).
  • So not all people were happy with how they were conducted.
  • The problem is the lack of time and the lack of substance.
  • Second and third person to review just rubber stamp the first code reviews.
• Energy
  • Who is doing the review?
  • How much time are they spending on it?
  • “Everyone on the team should get the same review.” “I feel that senior people don’t receive much feedback since they can ‘do no wrong’.” “Code reviews need to be acknowledged as a first-class citizen.”
  • Developers are frustrated by big commits, large PRs, and no context.
  • Unequal reviews (Jr vs. Sr.)
  • Frustration: When CTO/senior devs would treat code review as second-class citizens.
  • What people are saying and doing when reviewing is pretty important.
  • 5% of people just have an automatic negative response to code review.
  • People just did not like tons of comments on a pull request. Better to have a conversation. Also, style/syntax vs content/semantics.
  • People who had empathetic code reviews have better experiences than people who have egotistical code reviews. You should review the code, not the person, just let an automated tool fix the small errors.
  • 25 years ago, there was no concept of checklists on what code reviews should have or have not. People are just going through the motions.
  • Honestly we aren’t sure if we are doing the code reviews right, but at least we are talking about it a lot.
• Small wins
  • Commit hooks are awesome.
  • PR templates!
  • Screenshots/gifs.
  • Linters.
  • Encapsulate everything: small commits, small pull requests, make things easier to review.
  • Specific reviewers.
• Big wins.
  • Review everyone: push for a culture that values vulnerability. Make sure that everyone puts themselves out there. Senior developers can make mistakes, and it’s important for junior developers to see that.
  • Call out: but call out the good stuff too!
  • Iterate: start the conversation if you feel the code review process is where you want to be.
• “In theory, I love code reviews. In practice, they are only as good as the group that’s responsible for conducting them in the right manner.”
• Pair programming: Yeah
• Checklist: Can I read this code, if someone left the company and someone else looked at the code, review. Even if it’s a bug fix, there should be a test for the fix for the bug fix.
• http://danger.systems/.

Compacting GC in MRI, Aaron Patterson

• I work for a Le Git company. I forked Matz’s Le JIT on Le Git.
• Planck mechanical keyboard, LTrac mouse, Norman layout.
• Compact GC
  • Ned to have objects, and free space for the objects to compact into.
  • Allocation: This is the GC’s responsibility. Apparently GC is also a garbage producer.
  • rb_newobj in gc.c.
  • Heap -> page -> object, creating a page is slower than just finding an empty object slot. You want to reduce the number of slow path slots (RUBY_GC_HEAP_FREE_SLOTS).
  • Pages = “arenas”.
  • Slots are fixed width (40 bytes). So how do we store big hashes which are more than 40 bytes? What we do an st_hash, allocated not on the hash, and we do this with MALLOC. When we free this st_hash, we also free the allocated hash.
  • We can control the layout and the format of the GC, but the ones outside the system allocations, we can’t control where those are.
  • heap has_many pages, page has_many slots, slot.empty? or slot.has_object?
• Mark/Sweep
  • We start from the root, walk through the tree, and anything that isn’t free is conserved and those.
  • The code to do this is gc_mark. Either incremental or lazy sweeping.
  • Compacting: The amount of memory consumed is still the size of the heap or pages, even if the spaces for the object are unfilled. If we compact things, then we just rearrange objects to go to the pages with spaces, where we can then remove the excess pages. Problem is we can’t move the objects.
  • Compaction algorithm we use: 2 finger compaction. The problem with this is not that fast, and assigns objects randomly. The good thing is it is simple to understand and it is simple to implement.
  • It works by having 2 fingers, one for free/one for
  • Problem in Ruby is when you move the objects, the objects have references to what will now be empty slots. We just run across the heap and update the references for each object.
  • Then, fill the slots with free.
  • The cost of the algorithm is that we have to move the objects, update the references, and we have to run across to fill the slots with free.
  • Problem is with object addresses. C thinks that these addresses are constant and will never change.

3 Problem Cases

• C code must rb_gc_mark objects/hashes to not move them. Objects that are held by C extensions use rb_gc_mark, but objects held by Ruby doesn’t.
• Hash tables. Hash tables/KV pairs, if the address changes, so does the hash (???) so the solution is don’t allow hashes keys to move.
• Problem if a C extension and a Ruby extension own an object, which happens in global variables.
• The general problem is we can only update Ruby objects to point to other Ruby objects.
• Improvement due to compaction: from 518 to 180 pages with free objects. Better for copy/write optimizations.
• Parent and child processes (Didn’t get it)
• What happened with compaction is we got just a 2% reduction in memory. So why? Poor compaction, or just a few things can be compacted.
  • So maybe GC is just a very small part of the overall memory that can be used? So even if we optimize this, then not that big effect.
  • Malloc stack logging: depending on the number, either the GC or the system will use a significant part of the memory.
• Why compaction: save memory n 2 ways: it frees up pages, + copy/write performance.
  • object ids when reassigning? Cache the id
  • Compaction done on startup.
  • Anonymous classes – also moved.