fedora 蓝色小药丸

THOTCON 0xA recap

Sessions https://www.thotcon.org/schedule.html

THOTCON 0xA had these parts relevant to me

— Visit the villages

Usually I can spend half an hour or more to work on something. Last year it was a car hack game, and this year there was a radio related booth. I am always interested in radio stuff, but never tried it. Now everything is digital, software based (SDR) so it feels even more complex. I wondered why people are still playing with that, when there are better ways to communicate. One of the booth hosts told me he is into radios because things like wifi or bluetooth are just radios, so working on radios helps to understand or work on those things.

Didn't have time or the fun to work on car hacking booth. Every time I feel I am an outsider, it takes a lot time to settle down and enjoy some puzzles or conversations. I don't work in security, just interested, and I know so little. Worse is that I only attend the first day without the party. I have to take my son to a mini maker faire at Schurz HS on Saturday. Have to persuade myself to buy next year's ticket, but it might be better if someone I know can attend too? 

— Luckily there is someone I know (and who is always kind to me) also attending. Watched how he and another colleague solved puzzles

Collapse )
fedora 蓝色小药丸

Cost of living 2019

Now first quarter is over..

Things you have to pay all the time

— Mortgage, $1427/mo because we have a small condo and pay the minimum

— HOA, $189/mo, also on the lower side

— Car loan, $211/mo; gas is about $40/mo ($2.9/gallon) because we drive so little (but everyday), insurance is $106/mo. So far the car did not require maintenance. Maybe we'd be better off using Uber, but nobody bothered to calculate it.

— Utilities, $220/mo, electricity is around $40, remaining is gas, too much for a too small room :( we use Nest (and 3 extra sensors) to keep the room at ~71F

— RCN internet, $46/mo, I used Google Fi and had to pay for device, $70/mo; wife uses T-mobile $35/mo. We are not on speaking terms so she does not need even the included 100min talk time.

Things you cannot really miss

— Charitable givings, $200/mo and not recurring. This year I had to skip CSO season tickets due to tight budget, so sorry no more donations from me..

— Healthcare, $900/mo, but it is a one time event. I can only see one or two similar events coming up, not in a hurry

— Groceries, $600/mo

— Education, $500/mo, and it is including skating for both JY and me ($101/season), his Ukulele enrichment class ($35/30min), his after school programs (chess and piano for this term), other CPD programs like swimming and drumming

— Restaurant, $260/mo, include coffee shops, since only I eat lunch out, and the cost of lunch has increased to around $11/meal

Things that are flexible

— Electronics, $120/mo, got a pair of AirPods recently

Collapse )
fedora 蓝色小药丸

Shutdown application context

Recent technical issue at work involves how a sprint-boot application shutdown itself. There are several types of applications we have over here, some are simply spring-boot web applications or scheduled applications, which should never shutdown. Some are SQS listeners that also do not shutdown. On the other hand, there are batch type of applications that should shutdown gracefully. And such batch type might be SQS listeners too. So someone built a SQS handler that can shutdown when after a given time there is no message, and another guy built an override handler that also shutdown when a message matches some criteria (it does not matter if message is handled, it seems nobody cares as the message will be retried in the next scheduled batch). And some batch simply shutdown after it is done. The technical issue was that, they refuse to shutdown. It turns out such application has defined DataSource bean using the commons-dbcp2 module. In there there is a thread pool that has to be destroyed by calling BasicDataSource.close(), which is automatically done by spring-boot if everything is configured as intended. But in one case, the DataSource is not defined as a bean, so its "close()" method is not called. The IDE was able to catch some errors but the stack trace was wrong — did not show the problematic DataSource but point to a different one. In another case, DataSource has JMX warnings after application shutdown, so instead of disabling JMX, developer added (destroyMethod="") to the bean annotation, which means the "close()" method is ignored by spring-boot. Finally, some application does not close its application context. It is fine to ignore the "close()" in most cases, as most applications are supposed to run forever. And if the context is closed for a web application, such application will shutdown immediately after startup. Having subtle difference in an often boilerplate "Main" class is not a good practice when there are so many modules to maintain.. and it caused problem. If developer has to call System.exit(0) that is a bad smell.

Collapse )
fedora 蓝色小药丸

#thotcon badge?

[root@m4700 ~]# esptool write_flash 0x00000 /home/yuan/tracking/thotcon0x9/tc0x9.bin 
esptool.py v2.3.1
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Warning: Could not auto-detect Flash size (FlashID=0x1851f, SizeID=0x1), defaulting to 4MB
Flash params set to 0x0240
Compressed 296480 bytes to 211973...
Wrote 296480 bytes (211973 compressed) at 0x00000000 in 18.8 seconds (effective 126.0 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

不知道为什么,Windows 下 curl 只下载到一半文件, esptool 的 exe 和 py 都报 timeout receiving header,只能看到板子上有灯在闪。 Linux 下一下子就刷完了。大概是因为转到 Linux 下尝试之前拨动了开关,我本来以为刷 flash 一定要把开关拨到 off

刚拿到的时候,TPP 那一页可以显示几行字,现在停在 "Loading" 不动弹。大概是坏掉了吧。
fedora 蓝色小药丸

Portfolio weighted averages

Last week I got an assignment to make a small Java library for some often used portfolio calculations. A portfolio is a collection of holdings, each holding has a weight, has multiple associated attributes. All data type is Double. The calculation involved are just sum and averaging, which is so easy in Java 8 Stream. So I said there is nothing to re-use about.. plus a very generic routine is hard to understand and use. As an example I hand crafted a "classify" method that accepts a Function<H,C> to find its classification, and also uses a Map<C,Double> as its internal state, a method to update this state, a method to export a result. And that method internally calls another GenericAccumulator that simply iterates input. Long/Short handling was added as part of the iteration loop, and sum/averaging on the other hand was part of the "classify" method body. This successfully confused everyone including myself.

Collapse )
fedora 蓝色小药丸

Installed nvidia driver

Since Fedora 28 wayland is crashing for me (Dell m4700 with external monitor), I got some time to try the nvidia GPU driver. At first I looked at https://www.if-not-true-then-false.com/2015/fedora-nvidia-guide/ which shows the actual output when installed: in "About" page it will show the nvidia card name. Then since it is easier to install from RPM Fusion, I followed https://rpmfusion.org/Howto/NVIDIA. The document is concise but helpful. For example, when it says "Secure Boot" has issue, then it is best turned off in BIOS. For another example, when it says "Wayland" has issue and must install something from Copr, indeed that is the case. Also the "grubby" command to update kernel command line is helpful too.

I followed another article https://gorka.eguileor.com/vbox-vmware-in-secureboot-linux/ to sign the modules. First create a key, then register the key to UEFI, I never did this before. I cannot find the keyring ".system_keyring" but /proc/keys shows something else.

Collapse )
fedora 蓝色小药丸

Terraform tips (6)

To create resource without further updating it, use "ignore_changes" lifecycle property.

There are two cases we used this as a workaround. First, the lambdas are created with Terraform, but the code and configuration updates are in separate process. To prevent Terraform from overwriting code, the source_code_hash property can be ignored.

Another one is aws_lambda_alias. The issue is also caused by the 2-step process, that a version cannot be published until the last moment. Fortunately, the "function_version" can be ignored.

Another tip I want to mention is to read the code of verified modules. As I said in a previous post, Terraform lacks macros, so everything is repeated. And defining resource in modules is hard to manage. But a well written module seems to work. A module typically only defines one core resource, like one lambda or one s3 bucket. It might be overkill to wrap into a module, but writing shell scripts to generate code is not fun. What shell scripts can do, while module cannot, is optional properties that requires a value.

fedora 蓝色小药丸

Terraform tips (5)

The AWS resource is not applied in real-time. For some, the provider can wait for resource created, or poll the resource until it is created. But even provider and the service agreed, another resource may still hold an older view.

The document calls it "eventual consistency". The end result is that some "tf apply" may fail first but become successful in a second or later run.

Two resources have this issue so far:

— IAM role and policy attachment. When created role, attaching policy usually also works, but a user of the policy will likely see a role created but policy is not yet attached. The user should have "depends_on" to the attachment, and the attachment should have some sleep. Currently I set it to sleep 15s, which is acceptable.

— cloudwatch alarms also depends on policy being created. Similarly add depends_on and add 15s sleep. There is a bug reporting that policy not created after the target is re-created, and the workaround was to use interpolation in policy's name. But then that ticket did not mention the sleeping part.

fedora 蓝色小药丸

Terraform tips (4)

1. generate files for providers and for backend

provider file need to set provider version, and the content will be the same in all "root" modules. It is best to not to repeat oneself, checking-in the same file multiple times. It is not javascript..

similarly the backend is defined in a file that is best generated too. One reason is different use case requires different configurations -- automation has different AWS credentials, while running locally uses a different state file path and credential parameters; manually modify files in order to run it is too expensive. Also it prevents from mistakes manually updating this file. Another is that since files can be generated, it is easy to substitute a state file path.

2. generate files for different environments

same as #1, variable values can be generated. Settings like "vpc" can be defined in one place then get substituted and copied to the "root' module folder to use, save a lot of duplicate and manual works.

3. generate unimportant resources

since there is a chance to run shell scripts to generate files, why not to generate everything..

actually it is because Terraform lacks the ability to define any macros. A resource must exist or to be created, one cannot create a different number of resources on different environments. Suppose a list of s3 bucket names is given and Terraform has to repeatedly define those resources.. even one can use "count" as a workaround, the resource name will be awkwardly hard to use.

Collapse )
fedora 蓝色小药丸

Terraform tips (3)

(It will be shorter afterwards since I don't know much, just used it frequently recently.)

1. add alias

in .bashrc, alias "terraform" to "tf" which is the file extension. It saves a lot of typing

2. input, output and local variables

when declare a variable as "var", it is an "input" variable. And when declare a variable as "output", literally it is that. They are _not_ for debug purpose. The input and output is relative to a module.

And then "locals" defines variable in tfvars format. It is even not taught in detail in documentation, but it is the most useful feature. The reason is that "var" cannot use interpolation, while "output" cannot be referenced in current module. The only way to define something useful is through "locals".

3. modules and root module

A folder can be loaded as a module. There is only one "root" module when terraform executes. And for each module, terraform will load all files in that folder. Recognized files are

— all .tf files

— all .auto.tfvars files

— I name the generated files ".auto.tf", which follows the first rule.

— special file terraform.tfvars. I avoid this special file at all -- put it in .gitignore just in case. When I have to use a tfvars file, for the generated files I name it ".auto.tfvars" which follows the second rule. For different environment I may also define "terraform.PROD.tfvars" which _won't_ get loaded until one uses the "-var-file" parameter.

Collapse )