8 questions to ask yourself before diving into a new task
Most of the researches regarding the cost of software errors, show that there is an order of magnitude difference between the stages, where the error is found. That’s why it’s essential to take your time during the planning and prepare a detailed implementation plan for the more complex tasks. I created a personal checklist, which I use before starting a new task. Some of the steps may seem rather obvious, but it’s always worth to keep them in mind. Let’s grab a cup of coffee and enjoy the lecture :).
The Jungle Book
Recently I’ve found an interesting problem on HackerRank:
There are a number of animal species in the jungle. Each species has one or more predator that may be direct or indirect. Species X is said to be a predator of species Y if at least or of the following is true:
- Species X is a direct predator of species Y
- If species X is a direct predator of species Z, and Z is a direct predator of Y, then spec is an indirect predator of species Y.
Indirect predation is transitive through any numb of levels. Each species has a maximum of 1 direct predator. No two species will ever be mutual predators, and no species is a predator of itself. Your task is to determine the minimum number of groups that must be formed to so that no species is grouped with its predators, direct or indirect. As an example, consider an array where each position represents a species and each element represents a predator of that species or-1 if there are none. The array is a [-1, 8, 6, 0, 7, 3, 8, 9, -1, 6, 1] and we’ll use zero indexing.
0 8
\ / \
3 1 6
\ / / \
5 10 2 9
\
7
\
4
These are the trees created based on the given array.
The groups will be: [0,8], [3,1,6], [5,10,2,9], [7], [4]We can notice that the asked number of groups without predators is, in fact, the depth of the longers tree branch. Let’s move on directly to the solutions:
Good will hunting
Today was a sunny day, the one that makes you thrive for a beer, mint lemonade or an intergalactic crash, just like the one ending von Trier’s Melancholy. And in the middle of the heat, meditating like Kirsten Dunst on a green meadow, something just struck me. An idea.
Maybe partly it was caused by the third episode of the new Black Mirror season I’ve seen last night. It wasn’t really up to the standard the viewers were used to, myself included. Frankly, the plot was dry as a salt lake, the distance in quality from San Junipero should be expressed in light years, and it was night and night even compared to Hang the DJ. Just a teenage flick about bloodsucking business which creeps everywhere where big money is put at stake. To make thing interesting it gravitated around Miley Cyrus’ career…
How to sort an array of objects by a property value with priority in JavaScript?
User case: data should be sorted by custom values, which don’t make logical order. Look at the following data:
const list = [
{ category: 'tv', price: 1400 },
{ category: 'smartphones', price: 590 },
{ category: 'notebooks', price: 1500 },
{ category: 'smartphones', price: 350 }
]We would like to sort the objects in the following order: smartphones, notebooks, anything else and have no control over the order data is fetched. First, we need to create an object with that is going to hold priorities:
let sortingOrder = {
'smartphones': 1,
'notebooks': 2
}We are going to use standard array sorting method from the prototype, but need to write a comparison function. Let’s make it take the property name and sorting order as parameters, so it can be reused.
function compare(key, order = 'asc') {
return function (a, b) {
if (!a.hasOwnProperty(key) || !b.hasOwnProperty(key))
return 0;
const first = (a[key].toLowerCase() in sortingOrder) ? sortingOrder[a[key]] : Number.MAX_SAFE_INTEGER;
const second = (b[key].toLowerCase() in sortingOrder) ? sortingOrder[b[key]] : Number.MAX_SAFE_INTEGER;
let result = 0;
if (first < second)
result = -1;
else if (first > second)
result = 1;
return (order === 'desc') ? ~result : result
};
}We use the comparator function to determine the order from keys. Number.MAX_SAFE_INTEGER is used to be sure note specified elements are always last in undetermined order and we are free to extend sortingOrder object. Finally, (order === ‘desc’) ? ~result : result inverts the comparison result for descending order. Lastly, we sort the array as follows:
list.sort(compare('category'));
// OR
list.sort(compare('category', 'desc'));SQL ordered insert from bulk
Bulk insert is one of the best SQL features whenever performance is needed. It’s simple and straightforward, the order of data in a table is retained from the file. However, what if we have already defined import files and would like to feed data from the bulk table in an ordered manner, without modifying files’ structure? To be sure that the ordered will be kept we need to somehow introduce a sorting column.
Io and behold, 3 simple ways to import from temp table and keep entries’ order.
Bulk insert to a view
One way to do this is to modify the original bulk table by adding a primary key. We should always add it in the first position – for two reasons: clarity and ability to extend table columns without dropping whole table.
Note: We’re going to use the simplest example as possible for readability.
First, we create/alter the import table by adding a primary key, which later will be used to determine the order of the insert.
CREATE TABLE ImportTemp(
iId int IDENTITY(1,1),
[City] nvarchar(30)
)Then we create a view that contains all columns except identity.
CREATE VIEW ImportTempView
AS
SELECT
city
FROM ImportTempFinally simply perform the bulk insert to the view, results will be automatically inserted to the table on which view is based on.
BULK INSERT ImportTempView
FROM 'Q:\cities.txt'; | iId | City |
| 1 | Toronto |
| 2 | New York |
| 3 | Cracow |
| 4 | Warsaw |
Disadvantages?
It requires diving into the code base and adjusting the table, which is targeted during the import. Also, we add an additional abstraction layer plus performance overhead with the view itself.
SqlBulkCopyOptions
We’re going to use the .NET SqlBulkCopyOptions to specify the way import feeds identification data. However, it requires modification of the import files, which can disqualify this solution (no access to the import files, too many templates, etc.).
Example:
public SqlBulkCopyAssistant CreateBulkCopy(string strDestinationTable)
{
return CreateBulkCopy(strDestinationTable, SqlBulkCopyOptions.KeepIdentity);
}The SqlBulkCopyOptions.KeepIdentity option when turned on will allow feeding IDs from the import file. However, the import file needs to include ID values. If this option is disabled (bear in mind the SqlOptions use bitwise options), import file STILL needs to has the ID column specified, although it can be empty (letters in bold are just a header):
id, city
,toronto
DataTable and DataColumn
The third and final method is based on .NET methods involving operations on DataView and DataTables.
using (var dataView = _csvDataReader.ReadDataWithHeaders(localFilePath))
{
var dataTable = dataView.ToTable();
dataTable.Columns.Add("iId", typeof(int)).SetOrdinal(0);
var dataReader = dataTable.CreateDataReader();
_bulkUploadService.Upload(ProcessImport.LoadTempTableName, dataReader);
}Please focus on dataTable.Columns.Add(“iId”, typeof(int)).SetOrdinal(0); line, which is responsible for adding an empty column to the import table in the first position (0). This way we’re going to add a temporary dummy column on the fly and keep import clean without touching import files.
Whichever method you choose, you’ll end up having a temp table with identity column, which is both trivial and efficient to sort.
The Shallow – Nicolas G. Carr
I came by this book by accident. Found it laying on a seat in a coach going to the Italian Alps. I decided to give it a try for two reasons. Firstly, I’ve heard about the book and it struck me it was the finalist of the Pulitzer Prize. Secondly, I noticed increasing difficulty in maintaining attention while reading myself. What I hoped to be a winter romance, ended up as a single serving friend, leaving rather mixed feelings.
Why? The topic itself is really interesting and more current than ever. However, the book seems like an essay extended to absurd lengths. Let me take up the challenge and present all the key ideas from The Shallow in brief nine sentences. Here we go…
The book
In the history of mankind, there were three inventions that overhauled the way we think: the clock, the print, and the web. The human brain is neuroplastic and flexible – the most often we use any particular area, the more it grows its neural connections. Additionally, in case of emergency (accident, surgery), some parts can take responsibility for the others. Because of the amount of information we receive nowadays (study from 2009 showed about 34GB a day) we can notice a shift in the way of thinking. We tend to move from deep, focused and thoughtful sessions to multitasking and information skimming. This drove us toward difficulty with maintaining attention in situations that take more time and don’t flourish with stimulation ex. reading longer text forms. Finally, we started treating smartphones as extensions of our brain. It lets us store data, notifications, dates, guide us through a new town and much more. Since it reliefs brain from those tasks, it has a side effect of decreasing neural connections, which leads to difficulty in memorization, disorientation etc.
And there it is… I admit Carr did the homework and well documented every theory with convincing levels of example, mostly scientific research. Surely the way people interact with the Internet has changed and treating it as air, from which we breathe information may result in creating a shallower form of intelligence. The curse of abundance – too much data, too quickly, too divergently, too distractedly, too unassociatively, and with too much finality.
Conclusion
Reading The Shallow in 2019 surely won’t open your eyes. The theories became facts. So does the book became obsolete? Yes and no. Yes, because most of the information depicted in The Shallow as shocking is now taken for granted. And no – it’s more actual than ever as we tend to integrate stronger and stronger with mobile devices, constantly being attached to the web. Is this a bad thing though? It doesn’t make much sense to answer that question, it’s just inevitable.
Pro Git – Scott Chacon, Ben Straub
Welcome to the first part of the book project series. Without further ado, let’s get started
Undoubtedly, there are many books about
I picked it a few weeks before the company I work at decided to move from the SVN to GIT, making what’s supposed to be a huge step forward. It turned out it really was.
Through the first 100 pages, Ben explains the concept of git, which is a decentralized versioning system and underlines what distinguishes it from other versions systems i.e. SVN, CVS, and Mercurial. Additionally, the author covers a number of topics: git customization, setting up a git server, or repository migration, which may help you during git initial setup. Chapters are well structured, precise and thrift. Concepts are explained in a clear manner and easy to understand. The book helps newcomers with getting to grips with the concept of distributed work using branches and provides several user stories, which cover most of the daily usages of the versioning system.
It’s a great chunk to start with in order to build a solid base of knowledge. Even readers who are already familiar with the git mechanics may find something interesting and non-trivial like chapters covering the git internals. After reading Pro Git I felt comfortable enough to navigate toward the git official documentation with a lot more courage.
Personally, the most important takeaway was the function called bisect, which I didn’t know existed before reading this book. It got into my mind so deep I decided to immediately put it into action. In case you are not familiar with it, here’s a short explanation from the official documentation:
This command uses a binary search algorithm to find which commit in your project’s history introduced a bug. You use it by first telling it a “bad” commit that is known to contain the bug, and a “good” commit that is known to be before the bug was introduced. Then
https://git-scm.com/docs/git-bisectgit bisectpicks a commit between those two endpoints and asks you whether the selected commit is “good” or “bad”. It continues narrowing down the range until it finds the exact commit that introduced the change.
Using it along automated tests allows to quickly locate the commit that broke it, which may really speed up your work.
Would I recommend the book? Definitely, just bear in mind you can really skip half of it, if you are not interested in the migration, server picking, choosing a communication protocol, or the git internals. Otherwise it’s well worth your time.
The Book Project
I must admit being busy with my new position as C# developer I haven’t spend much time on my blog lately. It’s a high time to change it. I decided to challenge myself with a task of reading and reviewing 12 books (+ extras) that are more or less related to programming. Based on the most recommended books for programmers, I’ve prepared a list and decided to see for myself if they’re worth reading. Except for reviews I’m going to work through a few examples from each book that I find the most significant.
Here’s the list:
- Clean Code – Robert C. Martin
- Pro Git – Scott Chacon, Ben Straub
- Head First Design Patterns – Eric Freeman
- 97 Things Every Programmer Should Know – Kevin Henney
- Introduction to Algorithms (parts)
- Cracking the Code Interview – Kitoyana
- How to Create a Mind – Ray Kurzweil
- The Shallow – Nicolas G. Carr
- Thinking, Fast and Slow – Daniel Kahneman
- Code Complete – Steve McConnell
- Clean Architecture – Robert C. Martin
- Refactoring – Martin Fowler
![Turn your Android smartphone into a Linux web server [Part 2]](http://prochal.com/wp-content/uploads/2017/09/image-1240x606.jpg)
Turn your Android smartphone into a Linux web server [Part 2]
Welcome to the second part of the tutorial, in which I’ll show you how to deploy a LAMP stack on your droid smartphone. Here’s the link to the previous part.
Rule of three
As far as I’m concerned there are three ways to launch a web server on the android device.
First and the most challenging is to install Linux natively. This would require native drivers for your smartphone, as the android layer would be wiped out totally. I’ve seen some examples of such ports of HTC devices, however, I was unable to find any materials for my Xperia or other brands. If you’ve heard about such projects, don’t hesitate and let me know.
Alternatively, you can install one of the web server apps, that emulate the environment and let you use the AMP stack such as PAW Server, KSWeb, BIT Web Server or any other. The advantage of this solution is that you don’t need to root your device. You can have a preconfigured environment waiting for deployment in a blink of an eye. The downside is that you don’t have the flexibility of Linux machine, ability to install any additional software, get the latest version of PHP, Apache2, SQL. Also, it lacks configuration options.
The third option and the one I went for is to use rooted device along with Linux Deploy, Busy Box (you’ll have this one already installed if you followed the first part of this tutorial) and Remote VNC for streaming the Linux desktop. Basically, it lets you create a virtual machine running a distribution of Linux of your choice. It seems to be the best of two worlds: you can use everything Linux has to offer and don’t have to worry about the drivers. Surely it won’t run so smoothly as a native installation would, but the performance is ok for most of the needs.
Deploying a distribution of Linux of your choice
First install BusyBox – it gives your phone access to the Linux commands required by the Linux Deploy virtual machine. Once you install it, you do not have to worry about it.
Open Linux Deploy and make sure it has administrator privileges. Now press the small download button in the top or bottom corner of the screen. This will take you to the options menu. Most of the settings here can be left as they are, you can choose the Linux distribution. Everything is here – from Debian and Ubuntu to Kali Linux, Gentoo, Fedora and others. Each distribution has its own quirks and fans. One difference worth noticing is that there are two packages standards – DEB and YUM. In most cases it’s not a huge difference, however unless you own a Snapdragon 835 or newer, your software packages will be limited to the ones based on x64 libraries. For the sake of this tutorial, I went for Ubuntu which uses apt manager for DEB packages.
Next you should choose an installation directory and set the size of virtual machine image. This is important because by default the image size is set to 2GB, while the system files can take up to 1.6GB, leaving you with 400MB for your experiments. In case you use fat32 file system on your device/sdcard the max size you can use is roughly 4GB. If you’re lucky and your device is able to read ext4 cards, then you’ll be limited to 16TB per image or rather the card capacity.
Next under the GUI category tick enable under GUI to ensure you have a graphical interface to work with and check out the GUI Settings. Choosing the resolution is up to you. I’d suggest to set it to either your smartphone default resolution in case you’ll display it on a smartphone or to the native resolution of any external screen you want to use. Set your username to ‘root’ to give yourself privileged access and make a note of the password.
Now click the menu that looks like three dots and click ‘Install’. This will take a little while but will install the entire Linux distro on your device. Click the ‘Start’ button to run it in ‘chroot’ – essentially a small sub-compartment on your device. Finally, you’ll use VNC Viewer in order to actually view the GUI. Load it up and set the address to ‘localhost:5900’. Then enter the password that you set earlier. You can either install VNC on your smartphone or desktop just as you’d use any remote desktop. In case you’d want to stream the Linux desktop to your smartphone directly, you need a USB to MicroUSB OTG cable and a set of keyboard and mouse for convenience.
Installing LAMP
First, make sure you’re logged as root. If you set the user to root in the previous step, most likely you would. If not, elevate user privileges by adding ‘sudo’ word before all commands from this chapter. Next, ensure your system and apt package lists are fully up-to-date by running the following:
apt-get update -y && apt-get upgrade -y
The -y flag tells apt to go ahead and update the system without prompting you. Next you’ll need some kind of text editor. Personally I used nano, but actually any is going, to be fine. It’s up to your preference.
apt-get install nano -y
OR:
apt-get install gedit -y
PHP 7
apt-get install python-software-properties
add-apt-repository ppa:ondrej/php
apt-get update
apt-get purge php5-fpm
apt-get install php7.0-cli php7.0-common libapache2-mod-php7.0 php7.0 php7.0-mysql php7.0-fpm php7.0-mysql
apt-get install php7.0-cgi php7.0-dbg php7.0-dev php7.0-curl php7.0-gd
apt-get install php7.0-mcrypt php7.0-xsl php7.0-intl
After the installation is complete, restart Apache2:
service apache2 restart
And verify if the PHP7 installed correctly:
php -v
Apache2
Please run for the standard Apache installation
apt-get install apache2
or in case you don’t want to use the php-fpm package:
apt-get install apache2 libapache2-mod-php -y
Check if the Apache is running:
service apache2 status
If not, try to start it manually:
service apache2 start
Apache2 enable mod_rewrite
sudo a2enmod rewrite
sudo service apache2 restart
Then open the configuration file:
nano /etc/apache2/apache2.conf
AllowOverride None to AllowOverride All
<Directory /var/www/>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>sudo service apache2 restart
MySQL
apt-get install mysql-server -y
mysql_secure_installation
Follow the prompts as below:
- Would you like to setup VALIDATE PASSWORD plugin? press Enter here for No
- Please set the password for root here. New password: Type a secure password here then press Enter
- Re-enter new password: Re-type the secure password here then press Enter
- Remove anonymous users? (Press y|Y for Yes, any other key for No) : press y and then Enter here
- Disallow root login remotely? (Press y|Y for Yes, any other key for No) : press y and then Enter here
- Remove test database and access to it? (Press y|Y for Yes, any other key for No) : press y and then Enter here
- Reload privilege tables now? (Press y|Y for Yes, any other key for No) : press y and then Enter here
Then try to start mysql service:
service mysql start
You log into the client by running:
mysql -u root -p
To find port mysql is listening at just log into the client the type:
SHOW GLOBAL VARIABLES LIKE ‘PORT’;
If it works, well done! If not, check the error log in console or at /var/log/mysql/error.log. Some of the most frequent issues are:
- Cannot touch ‘/var/log/mysql/error.log’: Permission denied
This means the MySQL process cannot access logs directory, you can fix it by running:
chown -R mysql /var/log/mysql
…or any other mentioned catalog in the log. If that doesn’t help, try setting the access rights by running CHMOD -R 777 for given directory and gradually decrease the rights.
- ERROR: The partition with /var/lib/mysql is too full!
Check if you have any free space on the partition, if you do and the error persists, try changing the owner of the catalog recursively as in the previous example.
- ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (111)
Firstly, you can try going to /etc/mysql/my.cnf and change ‘bind-address’ parameter to ‘localhost’ or ‘127.0.0.1’. Sometimes the Android kernel can be compiled with CONFIG_ANDROID_PARANOID_NETWORK setting. This allows only certain users to use the device network i.e. users that belong to hardcoded groups –
aid_bt
aid_bt_net
aid_inet
aid_net_raw
aid_admin
To fix it add mysql user to one of the groups:
usermod -a -G aid_inet,aid_net_raw mysql
Well done, you have a working web server! 🙂 Just put your project in the var/www/html catalog or define your own in the etc/apache2/sites-enabled configuration file. To find your IP address, to which you can connect from outside devices, run the following command:
ifconfig | grep inet
Turn your Android smartphone into a Linux web server [Part 1]
Wait but Why?
My journey with the Coderslab coding boot camp is coming to an end. For the final project, I’ve decided to design an online film database parser, implementing RESTful API and turn it into a simple website. If you’re interested, please follow this link to Github, where you’ll find both project source files and detailed description. It should offer only a single functionality – option to select a random movie matching chosen criteria (been ever wondering what to watch this evening?). To spice it up slightly and make myself an opportunity to implement some SQL performance tuning, I decided to deploy the website on an experimental architecture.
My first thought went to Raspberry Pi due to its incredible fan base and an immense amount of high-quality tutorials. Then I remembered that I have a glossy orange Sony Xperia Z3 Compact in my drawer, which works perfectly fine despite screen issues. Android system is based on Linux, so I thought while being more challenging, turning the phone into mobile, waterproof (almost!), the battery-powered web server would be also more satisfying.
