15 Mar 2017
I’ve found many APK tools to be quite useful for diagnosing issues with Xamarin.Android. Here is a small list of tools that I find helpful.
apktool is a tool for reverse engineering Android applications. It can be used to decode resources to nearly original form and rebuild them after making some modifications.
Decode
One of the most useful usecases of apktool is to ensure your generated .apk is correct. You can use the decode / d method to take an .apk and “unzip” it’s original contents.
usage: apktool d MyApplication.apk
Build
The other usecase is to build a .apk after making a couple of modifications. You can use the build / b method to build an .apk
usage: apktool b foo
Which will build a folder named foo into an .apk file.
Note: If you aren’t using the wrapper script, you may need to invoke it via java -jar apktool.jar
aapt
aapt provides a couple of useful tools for our .apk files. You can find aapt inside of your Android SDK’s build-tools folder.
list
aapt list - shows the contents of the package.
usage: aapt list MyApplication.apk
META-INF/MANIFEST.MF
META-INF/ANDROIDD.SF
META-INF/ANDROIDD.RSA
AndroidManifest.xml
res/drawable/icon.png
res/layout/main.xml
resources.arsc
NOTICE
classes.dex
assemblies/MyApplication.dll
assemblies/Java.Interop.dll
assemblies/Microsoft.CSharp.dll
assemblies/Mono.Android.dll
assemblies/mscorlib.dll
assemblies/System.Core.dll
assemblies/System.dll
assemblies/System.Xml.dll
assemblies/System.Threading.dll
assemblies/System.Runtime.dll
assemblies/System.Collections.dll
assemblies/System.Collections.Concurrent.dll
assemblies/System.Diagnostics.Debug.dll
assemblies/System.Reflection.dll
assemblies/System.Linq.dll
assemblies/System.Runtime.InteropServices.dll
assemblies/System.Runtime.Extensions.dll
assemblies/System.Reflection.Extensions.dll
assemblies/Mono.CSharp.dll
assemblies/System.Runtime.Serialization.dll
environment
lib/armeabi-v7a/libmonodroid.so
lib/armeabi-v7a/libmonosgen-2.0.so
typemap.jm
typemap.mj
dump
aapt dump - provides values of individual elements or parts of a package. There are a few different items we can dump:
badging - dumps badging information like the versions, sdk targets, etc
usage: aapt dump badging MyApplication.apk
package: name='MyApplication.MyApplication' versionCode='1' versionName='1.0' platformBuildVersionName='7.1.1'
sdkVersion:'16'
application-icon-160:'res/drawable/icon.png'
application: label='' icon='res/drawable/icon.png'
launchable-activity: name='md5ef8aabbe49b48f4362a43abcfb4dce14.MainActivity' label='MyApplication' icon='res/drawable/icon.png'
feature-group: label=''
uses-feature: name='android.hardware.faketouch'
uses-implied-feature: name='android.hardware.faketouch' reason='default feature for all apps'
main
supports-screens: 'small' 'normal' 'large' 'xlarge'
supports-any-density: 'true'
locales:
densities: '160'
native-code: 'armeabi-v7a'
permissions - dumps permission information such as what uses-permissions are in use
usage: aapt dump permissions MyApplication.apk
package: MyApplication.MyApplication
uses-permission: name='android.permission.INTERNET'
uses-permission: name='android.permission.READ_CALENDAR'
uses-permission: name='android.permission.READ_CONTACTS'
resources - dumps resources in each resource category such as string/array/layout/etc
usage: aapt dump resources MyApplication.apk
Package Groups (1)
Package Group 0 id=0x7f packageCount=1 name=MyApplication.MyApplication
Package 0 id=0x7f name=MyApplication.MyApplication
type 1 configCount=1 entryCount=1
spec resource 0x7f020000 MyApplication.MyApplication:drawable/icon: flags=0x00000000
config (default):
resource 0x7f020000 MyApplication.MyApplication:drawable/icon: t=0x03 d=0x00000000 (s=0x0008 r=0x00)
type 2 configCount=1 entryCount=1
spec resource 0x7f030000 MyApplication.MyApplication:layout/main: flags=0x00000000
config (default):
resource 0x7f030000 MyApplication.MyApplication:layout/main: t=0x03 d=0x00000001 (s=0x0008 r=0x00)
type 3 configCount=1 entryCount=2
spec resource 0x7f040000 MyApplication.MyApplication:string/Hello: flags=0x00000000
spec resource 0x7f040001 MyApplication.MyApplication:string/ApplicationName: flags=0x00000000
config (default):
resource 0x7f040000 MyApplication.MyApplication:string/Hello: t=0x03 d=0x00000002 (s=0x0008 r=0x00)
resource 0x7f040001 MyApplication.MyApplication:string/ApplicationName: t=0x03 d=0x00000003 (s=0x0008 r=0x00)
configurations - dumps any configuration that has been setup
usage: aapt dump configurations MyApplication.apk
imsi=0/0 lang=-- reg=-- orient=0 touch=0 dens=160 kbd=0 nav=0 input=0 scrnW=0 scrnH=0 sz=0 long=0 vers=0.0
imsi=0/0 lang=-- reg=-- orient=0 touch=0 dens=240 kbd=0 nav=0 input=0 scrnW=0 scrnH=0 sz=0 long=0 vers=0.0
imsi=0/0 lang=-- reg=-- orient=0 touch=0 dens=0 kbd=0 nav=0 input=0 scrnW=0 scrnH=0 sz=0 long=0 vers=0.0
xmltree - dumps the xml parse tree for an xml file in a package
usage: aapt dump xmltree MyApplication.apk res/layout/main.xml
N: android=http://schemas.android.com/apk/res/android
E: LinearLayout (line=1)
A: android:orientation(0x010100c4)=(type 0x10)0x1
A: android:layout_width(0x010100f4)=(type 0x10)0xffffffff
A: android:layout_height(0x010100f5)=(type 0x10)0xffffffff
xmlstrings - dumps all strings inside for an xml file in a package
usage: aapt dump xmlstrings MyApplication.apk res/layout/main.xml
String pool of 7 unique UTF-8 non-sorted strings, 7 entries and 0 styles using 176 bytes:
String #0: orientation
String #1: layout_width
String #2: layout_height
String #3: android
String #4: http://schemas.android.com/apk/res/android
String #5:
String #6: LinearLayout
ClassyShark
ClassyShark is a binary inspection tool that can help you find information such as your total dex count, decompilation of .class files, and much more. It supports .dex, .aar, .so, .apk, .jar, .class, .xml files for further inspection.
usage: java -jar classyshark.jar
You should see a program like the following:
You can then inspect any aspect of your binary such as decompiling a class, viewing what’s inside of a dex file, and much more.
Classes
You can view what’s inside of your main dex list. Also known as the classes.dex file that is generated in the Android build process.
Methods Count
You can also see the current method count of the classes.dex list from an overview
Summary
Although this blog post only showcases 3 tools, there are so many ways to get information from your binaries to diagnose issues such as multidex, versioning, etc in you Xamarin.Android application!
If you enjoyed this post, please consider subscribing to my upcoming book’s email list:
You can signup at the following link: Programming Xamarin.Android Email List
07 Feb 2017
All the Android Things - Weather Station
When Android Things was first announced, my inner inventor came out. I wanted to start prototyping all of these ideas that have been spinning around my head. I wanted to be the next Red Green.
(If you aren’t familiar. He was the main character of The Red Green Show, which was a popular Canadian parody of a typical DIY/home improvement show. In the show, they would have some zany premise of automating things that are often painful like mowing your lawn, rolling down manual car windows, and serving beverages).
Android Things
Android Things provides the same Android development tools that we know and love to work with embedded devices. You can see a quick overview of this here:
There are two main components of the Things Support Library that we need to know before starting a prototype.
Peripheral I/O API
The Peripheral I/O APIs let your apps communicate with sensors and actuators using industry standard protocols and interfaces. The following interfaces are supported: GPIO, PWM, I2C, SPI, UART.
User Driver API
User drivers extend existing Android framework services and allow apps to inject hardware events into the framework that other apps can access using the standard Android APIs.
What you’ll need
Optional: You can also pick up a Pimoroni Rainbow HAT for a plug-and-play buffet of inputs, sensors, etc to explore Android Things.
Getting Started
Once you’ve gathered your materials, you’ll need to flash your device. After you’ve flashed your device, you will need to setup a network connection. You can follow the various instructions for doing this here:
Set Up Your Development Environment
After you’ve flashed your device and have it connected to your network, we want to ensure it’s available via adb. You can use the following command to ensure this:
adb devices
When you bootup the Android Things device, you can either have it connected to an ethernet cord, or to your wireless network. Either way, you will want to use adb connect <ip address> to connect to the device.
Android Weather Station Sample
This sample is an integration of multiple peripheral sensors to analyze and display current weather information.
You can find the source code of this sample here:
https://github.com/JonDouglas/XamarinComponents/tree/master/Android/AndroidThingsContribDrivers/samples/WeatherStation
Sample
Monitor
Displays the current weather situation. Sunny, Cloudy, or Rainy.
Raspberry Pi 3
Displays the current realtime temperature in Celsius. Here’s a video of it in action:
https://www.screencast.com/t/i3vaGchmH7
Summary
Android Things is really fun to tinker around with. Let your inner Red Green or Tim “The Tool Man” Taylor shine!
01 Feb 2017
Xamarin.Android Book
There’s been one thing that Xamarin developers have repeatedly told me in the past:
You should write a book with all of the knowledge you have.
I’ve finally taken this into consideration and decided that I will be writing a complete
book on Xamarin.Android.
I hope to help you with the challenges of Android app development using the Xamarin.Android framework whether
you are new to app development or already nose deep into a project.
This book will be much different than most Xamarin books in the sense that it will focus purely on the Xamarin.Android
platform.
Cover
Subscribe to the email list
One thing I would like to ask each one of you is to subscribe to my email list to get updates on the book and early access to various chapters.
I’ve added a field for your feedback as to what you want to see covered in more depth.
You can signup at the following link: Programming Xamarin.Android Email List
26 Jan 2017
NOTE: This blog was featured on blog.xamarin.com in which you can view a condensed version here: https://blog.xamarin.com/tips-for-creating-a-smooth-and-fluid-android-ui/
As developers, we want our users to have buttery smooth experiences when using our application. When it comes to UI performance, a buttery smooth experience can be defined as a consistent 60 frames per second(fps). That means that we have to render a frame every 16 ms to achieve this experience.
Note:
You can exceed the 16 ms time to render a frame occasionally, as there are often one or two buffered frames ready to go.
Why 60 fps?
Watch this wonderful video by Google on this topic for a brief introduction:
This gives our whole frame process a little less than 16 ms to fully draw a frame on the screen. If a frame is dropped or delayed, it is known as Jank.
Identifying the problem
Have you ever had an experience like this with your application? Your scrolls are very choppy and your UI doesn’t seem to be that responsive.
Credit: Doug Sillars https://github.com/dougsillars for a great example.
During the time that the application is unresponsive and choppy, if the user attempts to interact with the application and it takes longer than 5000 ms (5 seconds) for the application to respond, the Android OS will give us a lovely message:
This isn’t the best experience we can offer our users.
Digging Deeper
Have you ever seen the following in your adb logcat?
I/Choreographer(1200): Skipped 60 frames! The application may be doing too much work on its main thread.
This is the Android Choreographer warning you in advance that your application is not performing to the buttery smooth experience we are aiming for. In fact it’s telling you that you are skipping frames and not providing your users a 60 FPS experience. We will need a few tips & tricks to resolve these issues.
- Measure overall UI performance with Systrace to get a baseline.
- Use Hierarchy Viewer to identify and flatten view hierarchies.
- Reduce overdraw by flattening layouts and drawing less pixels on screen.
- Enable StrictMode to identify and make fewer potentially blocking calls on the main UI thread.
Getting Started
Counting Janky Frames
First you’ll want to get an overview of the total janky frames. After your application has been running and interacted with to reproduce the jank, run the following command:
adb shell dumpsys gfxinfo <PACKAGE_NAME>
Sample Output:
Stats since: 524615985046231ns
Total frames rendered: 8325
Janky frames: 729 (8.76%)
90th percentile: 13ms
95th percentile: 20ms
99th percentile: 73ms
Number Missed Vsync: 294
Number High input latency: 47
Number Slow UI thread: 502
Number Slow bitmap uploads: 44
Number Slow issue draw commands: 135
Numbers never lie, so it’s apparent we have a bit of Janky frames(~9%). Let’s dig in further with Systrace.
Systrace
Systrace gives you an overview of the whole android system and tells you what’s going on at specific intervals of time.
Getting Started
Let’s start a systrace on our device. First open up Android Device Monitor to get started.
Once inside, we see the option to start a Systrace:
We then are prompted with what we would like to trace:
This will generate a trace.html file that will give us information about our system for the trace duration.
EX: Systrace over 30 seconds:
Okay great! But what does this all mean? Let’s take it a step at a time.
Alerts & Frames
Alerts will give you a description of what the current situation is with a respective frame(s). It might let you know that there was a long View.OnDraw() call and it might give you suggestions on how you can fix the relevant frame(s).
You can then dig straight into the frame.
And see how much time spent during each step
Finally you can mark that frame using the m hotkey and see what work is being done on various threads such as various CPU Threads, the UI Thread, and the RenderThread.
This example is showing a Yellow Frame, but we can get a general idea of what an idea performant Frame might look like.
From our Alert, we can see that we might want to avoid significant work in View.OnDraw() or Drawable.Draw(), especially allocations or drawing to Bitmaps. In other words, Google gives us a tip to watch this video:
For our Frame, we can see that our Adapter.GetView() should recycle the incoming View instead of creating a new one.
Systrace Terms
Frame Color
Green - Great performance
Yellow - Less than ideal performance
Red - Bad performance
Scheduling Delay
Scheduling delays happen when the thread that is processing a specific slice was not scheduled on the CPU for a long amount of time. Thus it takes longer for this thread to fire up and complete.
Wall Duration
The amount of time that passed from the moment a slice is started until it’s finished.
CPU Duration
The amount of time the CPU spent processing that slice.
Overdraw
Debugging overdraw is fairly easy. You can enable this in your Android device’s settings:
Settings -> Developer Options -> Debug GPU overdraw -> Show overdraw areas.
Once enabled, you will see many different colors on your layouts.
- White - No overdraw
- Blue - Pixels that are 1x overdrawn
- Green - Pixels that are 2x overdrawn
- Pink - Pixels that are 3x overdrawn
- Red - Pixels that are 4x overdrawn
Bad Layout Performance:
Good Layout Performance:
You can then identify why this layout might be overdrawing so much via a tool like Hierarchy Viewer.
Hierarchy Viewer
The first thing we want to know regarding our View Hierarchy is how deep or nested our layouts are.
Let’s take the previous example of bad layout performance:
We can see that we are 4 layers deep which is less than ideal for our ListView. We really need to flatten this out.
Okay that’s a little better! We are only 3 layers deep now. However it’s still not great. Let’s try to remove one more layer.
Much better! We just optimized our whole view hierarchy and we will reap the performance benefits. Let’s take a look at the overall Layout and Draw timings for proof.
- Bad Layout - 31 views / Layout: 0.754 ms / Draw: 7.273 ms
- Better Layout - 26 views / Layout: 0.474 ms / Draw: 6.191 ms
- Good Layout - 17 views / Layout: 0.474 ms / Draw: 1.888 ms
StrictMode
StrictMode is a very useful tool for battle testing your application. Enabling StrictMode as a means to ensure you are not putting extra work in certain places of your application like disk reads, disk writes, and network calls is ideal for a great user experience. In a nutshell StrictMode does the following:
- Logs a message to
LogCat under the StrictMode tag
- Display a dialog (If
PenaltyLog() is enabled)
- Crash your application (If
PenaltyDeath() is enabled)
This can help you determine what type of policies you’d like to battle test your application with.
Enabling StrictMode in your Android Application
protected override void OnCreate(Bundle bundle)
{
StrictMode.SetThreadPolicy(new StrictMode.ThreadPolicy.Builder().DetectAll().PenaltyLog().Build());
StrictMode.SetVmPolicy(new StrictMode.VmPolicy.Builder().DetectLeakedSqlLiteObjects().DetectLeakedClosableObjects().PenaltyLog().PenaltyDeath().Build());
base.OnCreate(bundle);
}
Summary
There are a plethora of tools available to use on your Xamarin.Android application. Use them to track the important performance-related items about your application such as rendering performance to achieve a buttery smooth 60 fps experience for your customers. Use tools like Systrace, GPU overdraw, Hierarchy Viewer, and StrictMode to pinpoint performance related issues in your application and fix them.
28 Dec 2016
Preface
Entity Framework has been one of my favorite projects for quite some time. If you didn’t know, Entity Framework split off from the 6.X version to a new re-written 7.X (Core) version, in which the goal is to keep the ORM lightweight and very extensible.
Now the challenge for me has always been:
I want to use Entity Framework in my Xamarin projects.
This hasn’t really been possible for the last 3 years. In fact, I’ve attempted it a few years back and gave up because the tooling was simply just not there. However this is my personal redemption at getting this to work.
Let’s talk about the tooling needed to make this possible. Consider this more of a high level approach:
- SQLite Provider- We will need a place to store our data. Seeing that SQLite is a perfect fit for mobile, we will use Entity Framework’s SQLite package:
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Sqlite/
- Entity Framework Core - We will need an ORM to manage our CRUD operations and database migrations:
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/
Great! Now we’re off to the races.
Getting Started
Creating the NetStandard library
The first thing we’ll do is create a netstandard library. To do this, create a PCL project:
Since this is not a netstandard library quite yet, let’s go ahead and convert that by going to the project’s Properties
Once we have that set, we want to ensure we have the minimum requirements for the two NuGet packages we linked above:
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Sqlite/
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/
By the looks of things, the dependency is netstandard 1.3. So let’s make sure our netstandard project is targeting that version:
Creating the Xamarin.Android project
Nothing too special here, we’re just going to create a File -> New Single-View App (Android) Project:
Adding the NuGet packages
This step is typically easiest to do on the project level rather than the solution level as NuGet is sometimes not that friendly with different project structures(.csproj vs project.json)
Adding the NuGet packages to the netstandard library
Adding the NuGet packages to the Xamarin.Android project
Defining the DbContext
If you’ve used Entity Framework before, you will be very familiar with how we define a DbContext and our underlying Models that define our database schema.
Let’s start with a simple data model that we will call Cat.cs:
public class Cat
{
[Key]
public int CatId { get; set; }
public string Name { get; set; }
public int MeowsPerSecond { get; set; }
}
Let’s now make sure that this is apart of our DbContext by defining a new context we’ll call CatContext.cs. You may notice that we have a string DatabasePath. We will use this later when we need to tell Entity Framework where to store our Database on disk:
public class CatContext : DbContext
{
public DbSet<Cat> Cats { get; set; }
private string DatabasePath { get; set; }
public CatContext()
{
}
public CatContext(string databasePath)
{
DatabasePath = databasePath;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlite($"Filename={DatabasePath}");
}
}
Implementing the Context
First, we need to make sure our Xamarin.Android project is referencing our netstandard library.
Now that we have that, let’s implement our MainActivity.cs with some Entity Framework code!
[Activity(Label = "EntityFrameworkWithXamarin.Droid", MainLauncher = true, Icon = "@drawable/icon")]
public class MainActivity : Activity
{
int count = 1;
protected async override void OnCreate(Bundle bundle)
{
base.OnCreate(bundle);
// Set our view from the "main" layout resource
SetContentView(Resource.Layout.Main);
// Get our button from the layout resource,
// and attach an event to it
Button button = FindViewById<Button>(Resource.Id.MyButton);
button.Click += delegate { button.Text = string.Format("{0} clicks!", count++); };
TextView textView = FindViewById<TextView>(Resource.Id.TextView1);
var dbFolder = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
var fileName = "Cats.db";
var dbFullPath = Path.Combine(dbFolder, fileName);
try
{
using (var db = new CatContext(dbFullPath))
{
await db.Database.MigrateAsync(); //We need to ensure the latest Migration was added. This is different than EnsureDatabaseCreated.
Cat catGary = new Cat() { CatId = 1, Name = "Gary", MeowsPerSecond = 5 };
Cat catJack = new Cat() { CatId = 2, Name = "Jack", MeowsPerSecond = 11 };
Cat catLuna = new Cat() { CatId = 3, Name = "Luna", MeowsPerSecond = 3 };
List<Cat> catsInTheHat = new List<Cat>() { catGary, catJack, catLuna };
if(await db.Cats.CountAsync() < 3)
{
await db.Cats.AddRangeAsync(catsInTheHat);
await db.SaveChangesAsync();
}
var catsInTheBag = await db.Cats.ToListAsync();
foreach(var cat in catsInTheBag)
{
textView.Text += $"{cat.CatId} - {cat.Name} - {cat.MeowsPerSecond}" + System.Environment.NewLine;
}
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
}
}
Let’s try to run this code.
Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'no such table: Cats'.
It looks like we’re missing a core Entity Framework feature, and that’s Migrations to create our database schema.
Well shucks…this is awkward. We don’t have a great way to generate Entity Framework Migrations from within a Xamarin.Android project or the netstandard library. Let’s work with a quick workaround by creating a new netcore Console Application so we can generate Migrations.
We need to add the Entity Framework Tools, Entity Framework Core Design, and Entity Framework Core to this project so we can use the command line to generate our Migrations.
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Tools.DotNet
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/
https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Design/
We now need to move over our Cat.cs and CatContext.cs to ensure there’s a DbContext it can generate Migrations for. Your Console App should now look like this:
Now we can generate a schema for our context. Let’s use the new dotnet tooling to do this. Open up a new console in our current Console App directory:
Now we need to generate an initial Migration.
dotnet ef migrations add Initial
This should generate a migration:
Now we need to take the initial migrations generated in the Migrations folder of our project and simply move them over to our netstandard library.
Note: You can simply change the namespaces of these two generated files to the name of your netstandard namespace.
Let’s try running the Xamarin.Android project again and see if we run into any other exceptions
It looks like it worked! Our simple attempt at adding Cat models and retrieving them works!
If we wanted to take a closer look at the SQLite file that gets generated, use an emulator and open up ADM(Android Device Monitor):
Taking a closer look into the data/data/files folder, we will see our Cats.db that we created.
You can now take that file and open it in any SQLite explorer.
Note: I personally use DB Browser for SQLite (http://sqlitebrowser.org/)
Conclusion
Source Code: https://github.com/JonDouglas/EntityFrameworkWithXamarin
It’s been a three year battle with you Entity Framework. However I have to give my thanks to everyone involved in getting Entity Framework to the state it currently is. It’s been so much fun seeing how these projects turn out after a long period of development.